2024-04-25T23:35:11.457 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T23:35:11.457 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --name teuthology-2024-04-25_21:00:03-rados-squid-distro-default-smithi --description rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} tasks/libcephsqlite} --archive /home/teuthworker/archive/teuthology-2024-04-25_21:00:03-rados-squid-distro-default-smithi/7673825 --owner scheduled_teuthology@teuthology -- /home/teuthworker/archive/teuthology-2024-04-25_21:00:03-rados-squid-distro-default-smithi/7673825/orig.config.yaml 2024-04-25T23:35:11.493 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T23:35:11.569 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-04-25_21:00:03-rados-squid-distro-default-smithi/7673825 branch: squid description: rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} tasks/libcephsqlite} email: ceph-qa@ceph.com first_in_suite: false job_id: '7673825' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: teuthology-2024-04-25_21:00:03-rados-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 - volumes: count: 4 size: 10 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: squid ceph: conf: client: debug cephsqlite: 20 debug client: 20 debug ms: 1 global: mon client directed command retry: 5 mon election default strategy: 3 ms bind msgr1: false ms bind msgr2: true ms inject socket failures: 1500 ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 mon scrub interval: 300 paxos service trim min: 10 osd: bluestore block size: 96636764160 bluestore compression algorithm: zstd bluestore compression mode: aggressive bluestore fsck on mount: true bluestore zero block detection: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd mclock override recovery settings: true osd mclock profile: high_recovery_ops osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - POOL_APP_NOT_ENABLED - do not have an application enabled sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: b22e2ebdeb24376882b7bda2a7329c8cccc2276a owner: scheduled_teuthology@teuthology priority: 101 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 9886 sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 sleep_before_teardown: 0 subset: 30270/100000 suite: rados suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_b22e2ebdeb24376882b7bda2a7329c8cccc2276a/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: b22e2ebdeb24376882b7bda2a7329c8cccc2276a targets: smithi057.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD6eQPdygOnUDfXan2c16/gwJdK6U1jFxx0VWVcM4g/QWx2r/ZEB85KlUChC3Ybyjvpw+DPEEc53uTveYuQ5jyM= smithi080.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM6Tgsrau0boLdwiy6/PvpPRBdo3WsqpxWOW+nDJOtz8Ev5gKHclznjvnCLL8KUm3vwA6SUZLB83ZqlgbG25tH0= tasks: - install: extra_system_packages: deb: - sqlite3 rpm: - sqlite-devel - ceph: null - exec: client.0: - ceph osd pool create cephsqlite - ceph auth get-or-create client.libcephsqlite mon 'profile simple-rados-client-with-blocklist' osd 'allow rwx pool=cephsqlite' >> /etc/ceph/ceph.keyring - exec: client.0: - ceph_test_libcephsqlite --id libcephsqlite --no-log-to-stderr - workunit: clients: client.0: - rados/test_libcephsqlite.sh cephsqlite env: CEPH_ARGS: --id libcephsqlite --no-log-to-stderr teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-25_21:00:03 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T23:35:11.569 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_b22e2ebdeb24376882b7bda2a7329c8cccc2276a/qa; will attempt to use it 2024-04-25T23:35:11.570 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_b22e2ebdeb24376882b7bda2a7329c8cccc2276a/qa/tasks 2024-04-25T23:35:11.570 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T23:35:11.572 INFO:teuthology.task.internal:Checking packages... 2024-04-25T23:35:11.596 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'd2c45d977142e024a59b68a8f1dce8f0b9840551' 2024-04-25T23:35:11.596 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T23:35:11.596 INFO:teuthology.packaging:ref: None 2024-04-25T23:35:11.596 INFO:teuthology.packaging:tag: None 2024-04-25T23:35:11.596 INFO:teuthology.packaging:branch: squid 2024-04-25T23:35:11.596 INFO:teuthology.packaging:sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T23:35:11.596 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=squid 2024-04-25T23:35:11.818 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2447-gb22e2ebd-1jammy 2024-04-25T23:35:11.820 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T23:35:11.826 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T23:35:11.826 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T23:35:11.843 INFO:teuthology.task.internal:Saving configuration 2024-04-25T23:35:11.860 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T23:35:11.867 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T23:35:11.886 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi057.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-25_21:00:03-rados-squid-distro-default-smithi/7673825', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 23:29:24.146051', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIP+Uytfa51Aah/aYr+Xeigm1H4bjhv+L590w/1HKIhMv'} 2024-04-25T23:35:11.909 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi080.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-25_21:00:03-rados-squid-distro-default-smithi/7673825', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 23:29:24.144617', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIFOMmAMitZ7XBFGNlRnuYVj30x6YwtFt1EeH9ABA2QuH'} 2024-04-25T23:35:11.910 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T23:35:11.916 INFO:teuthology.task.internal:roles: ubuntu@smithi057.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-04-25T23:35:11.916 INFO:teuthology.task.internal:roles: ubuntu@smithi080.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-04-25T23:35:11.916 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T23:35:12.029 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f3bc7291280>, signals=[15]) 2024-04-25T23:35:12.030 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T23:35:12.036 INFO:teuthology.task.internal:Opening connections... 2024-04-25T23:35:12.037 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi057.front.sepia.ceph.com 2024-04-25T23:35:12.040 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi057.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T23:35:12.121 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi080.front.sepia.ceph.com 2024-04-25T23:35:12.122 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T23:35:12.201 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T23:35:12.206 DEBUG:teuthology.orchestra.run.smithi057:> uname -m 2024-04-25T23:35:12.211 INFO:teuthology.orchestra.run.smithi057.stdout:x86_64 2024-04-25T23:35:12.211 DEBUG:teuthology.orchestra.run.smithi057:> cat /etc/os-release 2024-04-25T23:35:12.257 INFO:teuthology.orchestra.run.smithi057.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-25T23:35:12.257 INFO:teuthology.orchestra.run.smithi057.stdout:NAME="Ubuntu" 2024-04-25T23:35:12.257 INFO:teuthology.orchestra.run.smithi057.stdout:VERSION_ID="22.04" 2024-04-25T23:35:12.257 INFO:teuthology.orchestra.run.smithi057.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-25T23:35:12.257 INFO:teuthology.orchestra.run.smithi057.stdout:VERSION_CODENAME=jammy 2024-04-25T23:35:12.257 INFO:teuthology.orchestra.run.smithi057.stdout:ID=ubuntu 2024-04-25T23:35:12.257 INFO:teuthology.orchestra.run.smithi057.stdout:ID_LIKE=debian 2024-04-25T23:35:12.258 INFO:teuthology.orchestra.run.smithi057.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-25T23:35:12.258 INFO:teuthology.orchestra.run.smithi057.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-25T23:35:12.258 INFO:teuthology.orchestra.run.smithi057.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-25T23:35:12.258 INFO:teuthology.orchestra.run.smithi057.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-25T23:35:12.258 INFO:teuthology.orchestra.run.smithi057.stdout:UBUNTU_CODENAME=jammy 2024-04-25T23:35:12.258 INFO:teuthology.lock.ops:Updating smithi057.front.sepia.ceph.com on lock server 2024-04-25T23:35:12.296 DEBUG:teuthology.orchestra.run.smithi080:> uname -m 2024-04-25T23:35:12.300 INFO:teuthology.orchestra.run.smithi080.stdout:x86_64 2024-04-25T23:35:12.300 DEBUG:teuthology.orchestra.run.smithi080:> cat /etc/os-release 2024-04-25T23:35:12.345 INFO:teuthology.orchestra.run.smithi080.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-25T23:35:12.345 INFO:teuthology.orchestra.run.smithi080.stdout:NAME="Ubuntu" 2024-04-25T23:35:12.346 INFO:teuthology.orchestra.run.smithi080.stdout:VERSION_ID="22.04" 2024-04-25T23:35:12.346 INFO:teuthology.orchestra.run.smithi080.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-25T23:35:12.346 INFO:teuthology.orchestra.run.smithi080.stdout:VERSION_CODENAME=jammy 2024-04-25T23:35:12.346 INFO:teuthology.orchestra.run.smithi080.stdout:ID=ubuntu 2024-04-25T23:35:12.346 INFO:teuthology.orchestra.run.smithi080.stdout:ID_LIKE=debian 2024-04-25T23:35:12.346 INFO:teuthology.orchestra.run.smithi080.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-25T23:35:12.346 INFO:teuthology.orchestra.run.smithi080.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-25T23:35:12.346 INFO:teuthology.orchestra.run.smithi080.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-25T23:35:12.346 INFO:teuthology.orchestra.run.smithi080.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-25T23:35:12.346 INFO:teuthology.orchestra.run.smithi080.stdout:UBUNTU_CODENAME=jammy 2024-04-25T23:35:12.346 INFO:teuthology.lock.ops:Updating smithi080.front.sepia.ceph.com on lock server 2024-04-25T23:35:12.370 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T23:35:12.378 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T23:35:12.383 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T23:35:12.383 DEBUG:teuthology.orchestra.run.smithi057:> test '!' -e /home/ubuntu/cephtest 2024-04-25T23:35:12.386 DEBUG:teuthology.orchestra.run.smithi080:> test '!' -e /home/ubuntu/cephtest 2024-04-25T23:35:12.391 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T23:35:12.396 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T23:35:12.396 DEBUG:teuthology.orchestra.run.smithi057:> test -z $(ls -A /var/lib/ceph) 2024-04-25T23:35:12.435 DEBUG:teuthology.orchestra.run.smithi080:> test -z $(ls -A /var/lib/ceph) 2024-04-25T23:35:12.450 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T23:35:12.503 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T23:35:12.517 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-04-25T23:35:12.517 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-04-25T23:35:12.517 DEBUG:teuthology.orchestra.run.smithi057:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T23:35:12.518 DEBUG:teuthology.orchestra.run.smithi080:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T23:35:12.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T23:35:12.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T23:35:12.523 DEBUG:teuthology.orchestra.run.smithi057:> uname -r 2024-04-25T23:35:12.524 DEBUG:teuthology.orchestra.run.smithi080:> uname -r 2024-04-25T23:35:12.570 INFO:teuthology.orchestra.run.smithi057.stdout:5.15.0-56-generic 2024-04-25T23:35:12.570 INFO:teuthology.task.kernel:Running kernel on smithi057: 5.15.0-56-generic 2024-04-25T23:35:12.570 DEBUG:teuthology.orchestra.run.smithi057:> sudo apt-get clean 2024-04-25T23:35:12.571 INFO:teuthology.orchestra.run.smithi080.stdout:5.15.0-56-generic 2024-04-25T23:35:12.572 INFO:teuthology.task.kernel:Running kernel on smithi080: 5.15.0-56-generic 2024-04-25T23:35:12.572 DEBUG:teuthology.orchestra.run.smithi080:> sudo apt-get clean 2024-04-25T23:35:12.685 DEBUG:teuthology.orchestra.run.smithi057:> sudo apt-get update 2024-04-25T23:35:12.704 DEBUG:teuthology.orchestra.run.smithi080:> sudo apt-get update 2024-04-25T23:35:12.824 INFO:teuthology.orchestra.run.smithi057.stdout:Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-04-25T23:35:12.851 INFO:teuthology.orchestra.run.smithi080.stdout:Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-04-25T23:35:12.955 INFO:teuthology.orchestra.run.smithi080.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T23:35:12.955 INFO:teuthology.orchestra.run.smithi057.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T23:35:13.034 INFO:teuthology.orchestra.run.smithi057.stdout:Get:3 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,394 kB] 2024-04-25T23:35:13.036 INFO:teuthology.orchestra.run.smithi080.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-04-25T23:35:13.038 INFO:teuthology.orchestra.run.smithi057.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-04-25T23:35:13.071 INFO:teuthology.orchestra.run.smithi080.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [452 kB] 2024-04-25T23:35:13.151 INFO:teuthology.orchestra.run.smithi057.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [452 kB] 2024-04-25T23:35:13.155 INFO:teuthology.orchestra.run.smithi080.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,394 kB] 2024-04-25T23:35:13.156 INFO:teuthology.orchestra.run.smithi057.stdout:Get:6 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [243 kB] 2024-04-25T23:35:13.159 INFO:teuthology.orchestra.run.smithi057.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-04-25T23:35:13.159 INFO:teuthology.orchestra.run.smithi057.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,773 kB] 2024-04-25T23:35:13.183 INFO:teuthology.orchestra.run.smithi057.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [36.5 kB] 2024-04-25T23:35:13.184 INFO:teuthology.orchestra.run.smithi057.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [300 kB] 2024-04-25T23:35:13.187 INFO:teuthology.orchestra.run.smithi057.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-04-25T23:35:13.187 INFO:teuthology.orchestra.run.smithi057.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [848 kB] 2024-04-25T23:35:13.198 INFO:teuthology.orchestra.run.smithi057.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [601 kB] 2024-04-25T23:35:13.206 INFO:teuthology.orchestra.run.smithi080.stdout:Get:6 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [243 kB] 2024-04-25T23:35:13.207 INFO:teuthology.orchestra.run.smithi057.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [163 kB] 2024-04-25T23:35:13.208 INFO:teuthology.orchestra.run.smithi057.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-04-25T23:35:13.209 INFO:teuthology.orchestra.run.smithi057.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-04-25T23:35:13.209 INFO:teuthology.orchestra.run.smithi080.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-04-25T23:35:13.210 INFO:teuthology.orchestra.run.smithi057.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-04-25T23:35:13.210 INFO:teuthology.orchestra.run.smithi080.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,773 kB] 2024-04-25T23:35:13.230 INFO:teuthology.orchestra.run.smithi080.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [36.5 kB] 2024-04-25T23:35:13.231 INFO:teuthology.orchestra.run.smithi080.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [300 kB] 2024-04-25T23:35:13.235 INFO:teuthology.orchestra.run.smithi080.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-04-25T23:35:13.235 INFO:teuthology.orchestra.run.smithi080.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [848 kB] 2024-04-25T23:35:13.245 INFO:teuthology.orchestra.run.smithi080.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [601 kB] 2024-04-25T23:35:13.245 INFO:teuthology.orchestra.run.smithi057.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-04-25T23:35:13.245 INFO:teuthology.orchestra.run.smithi057.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-04-25T23:35:13.250 INFO:teuthology.orchestra.run.smithi080.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [163 kB] 2024-04-25T23:35:13.252 INFO:teuthology.orchestra.run.smithi080.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-04-25T23:35:13.252 INFO:teuthology.orchestra.run.smithi080.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-04-25T23:35:13.252 INFO:teuthology.orchestra.run.smithi080.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-04-25T23:35:13.283 INFO:teuthology.orchestra.run.smithi080.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-04-25T23:35:13.283 INFO:teuthology.orchestra.run.smithi080.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-04-25T23:35:13.410 INFO:teuthology.orchestra.run.smithi057.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-04-25T23:35:13.490 INFO:teuthology.orchestra.run.smithi080.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-04-25T23:35:13.552 INFO:teuthology.orchestra.run.smithi057.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [618 kB] 2024-04-25T23:35:13.719 INFO:teuthology.orchestra.run.smithi080.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,610 kB] 2024-04-25T23:35:13.756 INFO:teuthology.orchestra.run.smithi057.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,610 kB] 2024-04-25T23:35:13.914 INFO:teuthology.orchestra.run.smithi057.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [303 kB] 2024-04-25T23:35:13.928 INFO:teuthology.orchestra.run.smithi057.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-04-25T23:35:13.929 INFO:teuthology.orchestra.run.smithi057.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [38.1 kB] 2024-04-25T23:35:13.931 INFO:teuthology.orchestra.run.smithi057.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,830 kB] 2024-04-25T23:35:13.996 INFO:teuthology.orchestra.run.smithi057.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [311 kB] 2024-04-25T23:35:14.003 INFO:teuthology.orchestra.run.smithi057.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-04-25T23:35:14.205 INFO:teuthology.orchestra.run.smithi057.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [700 kB] 2024-04-25T23:35:15.793 INFO:teuthology.orchestra.run.smithi057.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,071 kB] 2024-04-25T23:35:16.870 INFO:teuthology.orchestra.run.smithi080.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [618 kB] 2024-04-25T23:35:17.911 INFO:teuthology.orchestra.run.smithi080.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [303 kB] 2024-04-25T23:35:17.922 INFO:teuthology.orchestra.run.smithi057.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [244 kB] 2024-04-25T23:35:18.247 INFO:teuthology.orchestra.run.smithi080.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-04-25T23:35:18.262 INFO:teuthology.orchestra.run.smithi080.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [38.1 kB] 2024-04-25T23:35:18.288 INFO:teuthology.orchestra.run.smithi080.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,830 kB] 2024-04-25T23:35:18.509 INFO:teuthology.orchestra.run.smithi057.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-04-25T23:35:18.564 INFO:teuthology.orchestra.run.smithi057.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,472 B] 2024-04-25T23:35:18.606 INFO:teuthology.orchestra.run.smithi057.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.7 kB] 2024-04-25T23:35:18.721 INFO:teuthology.orchestra.run.smithi057.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.4 kB] 2024-04-25T23:35:18.740 INFO:teuthology.orchestra.run.smithi057.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-04-25T23:35:18.740 INFO:teuthology.orchestra.run.smithi057.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.3 kB] 2024-04-25T23:35:18.882 INFO:teuthology.orchestra.run.smithi057.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.1 kB] 2024-04-25T23:35:19.032 INFO:teuthology.orchestra.run.smithi057.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-04-25T23:35:19.244 INFO:teuthology.orchestra.run.smithi057.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-04-25T23:35:19.326 INFO:teuthology.orchestra.run.smithi057.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [16.0 kB] 2024-04-25T23:35:19.408 INFO:teuthology.orchestra.run.smithi057.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [27.2 kB] 2024-04-25T23:35:19.489 INFO:teuthology.orchestra.run.smithi057.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-04-25T23:35:19.520 INFO:teuthology.orchestra.run.smithi057.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-04-25T23:35:22.129 INFO:teuthology.orchestra.run.smithi080.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [311 kB] 2024-04-25T23:35:22.318 INFO:teuthology.orchestra.run.smithi080.stdout:Ign:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en 2024-04-25T23:35:22.487 INFO:teuthology.orchestra.run.smithi080.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-04-25T23:35:22.488 INFO:teuthology.orchestra.run.smithi080.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,071 kB] 2024-04-25T23:35:24.222 INFO:teuthology.orchestra.run.smithi057.stdout:Fetched 13.2 MB in 7s (1,954 kB/s) 2024-04-25T23:35:25.193 INFO:teuthology.orchestra.run.smithi080.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [700 kB] 2024-04-25T23:35:25.639 INFO:teuthology.orchestra.run.smithi057.stdout:Reading package lists... 2024-04-25T23:35:25.656 DEBUG:teuthology.orchestra.run.smithi057:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-25T23:35:25.711 INFO:teuthology.orchestra.run.smithi057.stdout:Reading package lists... 2024-04-25T23:35:25.893 INFO:teuthology.orchestra.run.smithi057.stdout:Building dependency tree... 2024-04-25T23:35:25.894 INFO:teuthology.orchestra.run.smithi057.stdout:Reading state information... 2024-04-25T23:35:26.053 INFO:teuthology.orchestra.run.smithi057.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T23:35:26.053 INFO:teuthology.orchestra.run.smithi057.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T23:35:26.053 INFO:teuthology.orchestra.run.smithi057.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T23:35:26.054 INFO:teuthology.orchestra.run.smithi057.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T23:35:26.054 INFO:teuthology.orchestra.run.smithi057.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-04-25T23:35:26.054 INFO:teuthology.orchestra.run.smithi057.stdout: python2.7-minimal 2024-04-25T23:35:26.054 INFO:teuthology.orchestra.run.smithi057.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T23:35:26.055 INFO:teuthology.orchestra.run.smithi057.stdout:The following additional packages will be installed: 2024-04-25T23:35:26.055 INFO:teuthology.orchestra.run.smithi057.stdout: linux-generic linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-25T23:35:26.055 INFO:teuthology.orchestra.run.smithi057.stdout: linux-headers-generic linux-image-5.15.0-105-generic 2024-04-25T23:35:26.055 INFO:teuthology.orchestra.run.smithi057.stdout: linux-modules-5.15.0-105-generic linux-modules-extra-5.15.0-105-generic 2024-04-25T23:35:26.056 INFO:teuthology.orchestra.run.smithi057.stdout:Suggested packages: 2024-04-25T23:35:26.057 INFO:teuthology.orchestra.run.smithi057.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-04-25T23:35:26.057 INFO:teuthology.orchestra.run.smithi057.stdout:Recommended packages: 2024-04-25T23:35:26.057 INFO:teuthology.orchestra.run.smithi057.stdout: thermald 2024-04-25T23:35:26.126 INFO:teuthology.orchestra.run.smithi057.stdout:The following NEW packages will be installed: 2024-04-25T23:35:26.126 INFO:teuthology.orchestra.run.smithi057.stdout: linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-25T23:35:26.126 INFO:teuthology.orchestra.run.smithi057.stdout: linux-image-5.15.0-105-generic linux-modules-5.15.0-105-generic 2024-04-25T23:35:26.126 INFO:teuthology.orchestra.run.smithi057.stdout: linux-modules-extra-5.15.0-105-generic 2024-04-25T23:35:26.127 INFO:teuthology.orchestra.run.smithi057.stdout:The following packages will be upgraded: 2024-04-25T23:35:26.127 INFO:teuthology.orchestra.run.smithi057.stdout: linux-generic linux-headers-generic linux-image-generic 2024-04-25T23:35:26.185 INFO:teuthology.orchestra.run.smithi057.stdout:3 upgraded, 5 newly installed, 0 to remove and 315 not upgraded. 2024-04-25T23:35:26.185 INFO:teuthology.orchestra.run.smithi057.stdout:Need to get 113 MB of archives. 2024-04-25T23:35:26.185 INFO:teuthology.orchestra.run.smithi057.stdout:After this operation, 583 MB of additional disk space will be used. 2024-04-25T23:35:26.185 INFO:teuthology.orchestra.run.smithi057.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-105-generic amd64 5.15.0-105.115 [22.7 MB] 2024-04-25T23:35:26.690 INFO:teuthology.orchestra.run.smithi057.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-105-generic amd64 5.15.0-105.115 [11.5 MB] 2024-04-25T23:35:26.742 INFO:teuthology.orchestra.run.smithi080.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [244 kB] 2024-04-25T23:35:26.827 INFO:teuthology.orchestra.run.smithi057.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-105-generic amd64 5.15.0-105.115 [63.9 MB] 2024-04-25T23:35:27.431 INFO:teuthology.orchestra.run.smithi080.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-04-25T23:35:27.452 INFO:teuthology.orchestra.run.smithi080.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,472 B] 2024-04-25T23:35:27.455 INFO:teuthology.orchestra.run.smithi080.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.7 kB] 2024-04-25T23:35:27.542 INFO:teuthology.orchestra.run.smithi080.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.4 kB] 2024-04-25T23:35:27.554 INFO:teuthology.orchestra.run.smithi080.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-04-25T23:35:27.555 INFO:teuthology.orchestra.run.smithi080.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.1 kB] 2024-04-25T23:35:27.678 INFO:teuthology.orchestra.run.smithi080.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.3 kB] 2024-04-25T23:35:27.872 INFO:teuthology.orchestra.run.smithi057.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.105.102 [1,694 B] 2024-04-25T23:35:27.873 INFO:teuthology.orchestra.run.smithi057.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.105.102 [2,530 B] 2024-04-25T23:35:27.949 INFO:teuthology.orchestra.run.smithi057.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105 all 5.15.0-105.115 [12.3 MB] 2024-04-25T23:35:28.005 INFO:teuthology.orchestra.run.smithi080.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-04-25T23:35:28.088 INFO:teuthology.orchestra.run.smithi080.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-04-25T23:35:28.089 INFO:teuthology.orchestra.run.smithi080.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [27.2 kB] 2024-04-25T23:35:28.105 INFO:teuthology.orchestra.run.smithi057.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105-generic amd64 5.15.0-105.115 [2,857 kB] 2024-04-25T23:35:28.129 INFO:teuthology.orchestra.run.smithi057.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.105.102 [2,382 B] 2024-04-25T23:35:28.174 INFO:teuthology.orchestra.run.smithi080.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [16.0 kB] 2024-04-25T23:35:28.215 INFO:teuthology.orchestra.run.smithi080.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-04-25T23:35:28.246 INFO:teuthology.orchestra.run.smithi080.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-04-25T23:35:28.247 INFO:teuthology.orchestra.run.smithi080.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [311 kB] 2024-04-25T23:35:28.534 INFO:teuthology.orchestra.run.smithi057.stdout:Fetched 113 MB in 2s (56.8 MB/s) 2024-04-25T23:35:28.692 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package linux-modules-5.15.0-105-generic. 2024-04-25T23:35:30.632 INFO:teuthology.orchestra.run.smithi057.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-04-25T23:35:30.636 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../0-linux-modules-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T23:35:30.697 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T23:35:32.805 INFO:teuthology.orchestra.run.smithi080.stdout:Fetched 13.2 MB in 16s (837 kB/s) 2024-04-25T23:35:32.812 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package linux-image-5.15.0-105-generic. 2024-04-25T23:35:32.833 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../1-linux-image-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T23:35:32.926 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T23:35:33.325 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-105-generic. 2024-04-25T23:35:33.335 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T23:35:33.375 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T23:35:34.143 INFO:teuthology.orchestra.run.smithi080.stdout:Reading package lists... 2024-04-25T23:35:34.159 DEBUG:teuthology.orchestra.run.smithi080:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-25T23:35:34.219 INFO:teuthology.orchestra.run.smithi080.stdout:Reading package lists... 2024-04-25T23:35:34.420 INFO:teuthology.orchestra.run.smithi080.stdout:Building dependency tree... 2024-04-25T23:35:34.421 INFO:teuthology.orchestra.run.smithi080.stdout:Reading state information... 2024-04-25T23:35:34.583 INFO:teuthology.orchestra.run.smithi080.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T23:35:34.584 INFO:teuthology.orchestra.run.smithi080.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T23:35:34.584 INFO:teuthology.orchestra.run.smithi080.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T23:35:34.584 INFO:teuthology.orchestra.run.smithi080.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T23:35:34.584 INFO:teuthology.orchestra.run.smithi080.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-04-25T23:35:34.584 INFO:teuthology.orchestra.run.smithi080.stdout: python2.7-minimal 2024-04-25T23:35:34.584 INFO:teuthology.orchestra.run.smithi080.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T23:35:34.586 INFO:teuthology.orchestra.run.smithi080.stdout:The following additional packages will be installed: 2024-04-25T23:35:34.586 INFO:teuthology.orchestra.run.smithi080.stdout: linux-generic linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-25T23:35:34.586 INFO:teuthology.orchestra.run.smithi080.stdout: linux-headers-generic linux-image-5.15.0-105-generic 2024-04-25T23:35:34.586 INFO:teuthology.orchestra.run.smithi080.stdout: linux-modules-5.15.0-105-generic linux-modules-extra-5.15.0-105-generic 2024-04-25T23:35:34.587 INFO:teuthology.orchestra.run.smithi080.stdout:Suggested packages: 2024-04-25T23:35:34.587 INFO:teuthology.orchestra.run.smithi080.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-04-25T23:35:34.587 INFO:teuthology.orchestra.run.smithi080.stdout:Recommended packages: 2024-04-25T23:35:34.587 INFO:teuthology.orchestra.run.smithi080.stdout: thermald 2024-04-25T23:35:34.631 INFO:teuthology.orchestra.run.smithi080.stdout:The following NEW packages will be installed: 2024-04-25T23:35:34.631 INFO:teuthology.orchestra.run.smithi080.stdout: linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-25T23:35:34.631 INFO:teuthology.orchestra.run.smithi080.stdout: linux-image-5.15.0-105-generic linux-modules-5.15.0-105-generic 2024-04-25T23:35:34.632 INFO:teuthology.orchestra.run.smithi080.stdout: linux-modules-extra-5.15.0-105-generic 2024-04-25T23:35:34.632 INFO:teuthology.orchestra.run.smithi080.stdout:The following packages will be upgraded: 2024-04-25T23:35:34.633 INFO:teuthology.orchestra.run.smithi080.stdout: linux-generic linux-headers-generic linux-image-generic 2024-04-25T23:35:34.812 INFO:teuthology.orchestra.run.smithi080.stdout:3 upgraded, 5 newly installed, 0 to remove and 315 not upgraded. 2024-04-25T23:35:34.812 INFO:teuthology.orchestra.run.smithi080.stdout:Need to get 113 MB of archives. 2024-04-25T23:35:34.812 INFO:teuthology.orchestra.run.smithi080.stdout:After this operation, 583 MB of additional disk space will be used. 2024-04-25T23:35:34.812 INFO:teuthology.orchestra.run.smithi080.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-105-generic amd64 5.15.0-105.115 [22.7 MB] 2024-04-25T23:35:36.233 INFO:teuthology.orchestra.run.smithi080.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-105-generic amd64 5.15.0-105.115 [11.5 MB] 2024-04-25T23:35:36.585 INFO:teuthology.orchestra.run.smithi080.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-105-generic amd64 5.15.0-105.115 [63.9 MB] 2024-04-25T23:35:38.822 INFO:teuthology.orchestra.run.smithi080.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.105.102 [1,694 B] 2024-04-25T23:35:38.822 INFO:teuthology.orchestra.run.smithi080.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.105.102 [2,530 B] 2024-04-25T23:35:38.822 INFO:teuthology.orchestra.run.smithi080.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105 all 5.15.0-105.115 [12.3 MB] 2024-04-25T23:35:40.169 INFO:teuthology.orchestra.run.smithi080.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105-generic amd64 5.15.0-105.115 [2,857 kB] 2024-04-25T23:35:40.180 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../3-linux-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T23:35:40.289 INFO:teuthology.orchestra.run.smithi080.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.105.102 [2,382 B] 2024-04-25T23:35:40.290 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking linux-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T23:35:40.581 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T23:35:40.684 INFO:teuthology.orchestra.run.smithi080.stdout:Fetched 113 MB in 6s (20.1 MB/s) 2024-04-25T23:35:40.718 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking linux-image-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T23:35:40.865 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package linux-modules-5.15.0-105-generic. 2024-04-25T23:35:41.005 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package linux-headers-5.15.0-105. 2024-04-25T23:35:41.027 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../5-linux-headers-5.15.0-105_5.15.0-105.115_all.deb ... 2024-04-25T23:35:41.071 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-25T23:35:42.946 INFO:teuthology.orchestra.run.smithi080.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-04-25T23:35:42.949 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../0-linux-modules-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T23:35:43.012 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T23:35:45.138 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package linux-image-5.15.0-105-generic. 2024-04-25T23:35:45.159 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../1-linux-image-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T23:35:45.256 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T23:35:45.774 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-105-generic. 2024-04-25T23:35:45.795 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T23:35:45.833 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T23:35:47.153 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package linux-headers-5.15.0-105-generic. 2024-04-25T23:35:47.183 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../6-linux-headers-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T23:35:47.228 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T23:35:50.574 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T23:35:50.696 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking linux-headers-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T23:35:51.021 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-25T23:35:51.150 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T23:35:51.276 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up linux-headers-generic (5.15.0.105.102) ... 2024-04-25T23:35:51.404 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T23:35:52.732 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../3-linux-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T23:35:52.742 INFO:teuthology.orchestra.run.smithi057.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-04-25T23:35:52.743 INFO:teuthology.orchestra.run.smithi057.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-04-25T23:35:52.743 INFO:teuthology.orchestra.run.smithi057.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-105-generic 2024-04-25T23:35:52.743 INFO:teuthology.orchestra.run.smithi057.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-105-generic 2024-04-25T23:35:52.859 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking linux-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T23:35:52.897 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T23:35:53.209 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T23:35:53.337 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking linux-image-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T23:35:53.607 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package linux-headers-5.15.0-105. 2024-04-25T23:35:53.630 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../5-linux-headers-5.15.0-105_5.15.0-105.115_all.deb ... 2024-04-25T23:35:53.665 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-25T23:35:54.325 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T23:35:55.758 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up linux-image-generic (5.15.0.105.102) ... 2024-04-25T23:35:55.884 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up linux-generic (5.15.0.105.102) ... 2024-04-25T23:35:56.002 INFO:teuthology.orchestra.run.smithi057.stdout:Processing triggers for linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T23:35:56.061 INFO:teuthology.orchestra.run.smithi057.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-04-25T23:35:56.061 INFO:teuthology.orchestra.run.smithi057.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-105-generic 2024-04-25T23:35:59.766 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package linux-headers-5.15.0-105-generic. 2024-04-25T23:35:59.785 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../6-linux-headers-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T23:35:59.824 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T23:36:02.377 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T23:36:02.502 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking linux-headers-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T23:36:02.844 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-25T23:36:02.965 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T23:36:03.091 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up linux-headers-generic (5.15.0.105.102) ... 2024-04-25T23:36:03.219 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T23:36:04.659 INFO:teuthology.orchestra.run.smithi080.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-04-25T23:36:04.659 INFO:teuthology.orchestra.run.smithi080.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-04-25T23:36:04.659 INFO:teuthology.orchestra.run.smithi080.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-105-generic 2024-04-25T23:36:04.659 INFO:teuthology.orchestra.run.smithi080.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-105-generic 2024-04-25T23:36:05.172 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T23:36:06.573 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T23:36:07.982 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up linux-image-generic (5.15.0.105.102) ... 2024-04-25T23:36:08.108 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up linux-generic (5.15.0.105.102) ... 2024-04-25T23:36:08.234 INFO:teuthology.orchestra.run.smithi080.stdout:Processing triggers for linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T23:36:08.308 INFO:teuthology.orchestra.run.smithi080.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-04-25T23:36:08.308 INFO:teuthology.orchestra.run.smithi080.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-105-generic 2024-04-25T23:36:10.539 INFO:teuthology.orchestra.run.smithi057.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-04-25T23:36:10.540 INFO:teuthology.orchestra.run.smithi057.stdout:Sourcing file `/etc/default/grub' 2024-04-25T23:36:10.551 INFO:teuthology.orchestra.run.smithi057.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T23:36:10.576 INFO:teuthology.orchestra.run.smithi057.stdout:Generating grub configuration file ... 2024-04-25T23:36:10.825 INFO:teuthology.orchestra.run.smithi057.stdout:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-25T23:36:10.844 INFO:teuthology.orchestra.run.smithi057.stdout:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-25T23:36:11.127 INFO:teuthology.orchestra.run.smithi057.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T23:36:11.133 INFO:teuthology.orchestra.run.smithi057.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T23:36:11.205 INFO:teuthology.orchestra.run.smithi057.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T23:36:11.211 INFO:teuthology.orchestra.run.smithi057.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T23:36:11.382 INFO:teuthology.orchestra.run.smithi057.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T23:36:11.382 INFO:teuthology.orchestra.run.smithi057.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T23:36:11.382 INFO:teuthology.orchestra.run.smithi057.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T23:36:11.418 INFO:teuthology.orchestra.run.smithi057.stdout:done 2024-04-25T23:36:11.878 DEBUG:teuthology.orchestra.run.smithi057:> dpkg -s linux-image-generic 2024-04-25T23:36:11.895 INFO:teuthology.orchestra.run.smithi057.stdout:Package: linux-image-generic 2024-04-25T23:36:11.895 INFO:teuthology.orchestra.run.smithi057.stdout:Status: install ok installed 2024-04-25T23:36:11.895 INFO:teuthology.orchestra.run.smithi057.stdout:Priority: optional 2024-04-25T23:36:11.896 INFO:teuthology.orchestra.run.smithi057.stdout:Section: kernel 2024-04-25T23:36:11.896 INFO:teuthology.orchestra.run.smithi057.stdout:Installed-Size: 21 2024-04-25T23:36:11.896 INFO:teuthology.orchestra.run.smithi057.stdout:Maintainer: Ubuntu Kernel Team 2024-04-25T23:36:11.896 INFO:teuthology.orchestra.run.smithi057.stdout:Architecture: amd64 2024-04-25T23:36:11.896 INFO:teuthology.orchestra.run.smithi057.stdout:Source: linux-meta 2024-04-25T23:36:11.896 INFO:teuthology.orchestra.run.smithi057.stdout:Version: 5.15.0.105.102 2024-04-25T23:36:11.896 INFO:teuthology.orchestra.run.smithi057.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-25T23:36:11.896 INFO:teuthology.orchestra.run.smithi057.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-25T23:36:11.896 INFO:teuthology.orchestra.run.smithi057.stdout:Recommends: thermald 2024-04-25T23:36:11.896 INFO:teuthology.orchestra.run.smithi057.stdout:Description: Generic Linux kernel image 2024-04-25T23:36:11.896 INFO:teuthology.orchestra.run.smithi057.stdout: This package will always depend on the latest generic kernel image 2024-04-25T23:36:11.896 INFO:teuthology.orchestra.run.smithi057.stdout: available. 2024-04-25T23:36:11.896 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-105-generic 2024-04-25T23:36:11.897 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-25T23:36:11.897 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-25T23:36:11.897 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-04-25T23:36:11.897 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi057.front.sepia.ceph.com, path=None, version=distro) 2024-04-25T23:36:11.897 DEBUG:teuthology.orchestra.run.smithi057:> sudo apt-get clean 2024-04-25T23:36:12.036 DEBUG:teuthology.orchestra.run.smithi057:> sudo apt-get update 2024-04-25T23:36:12.154 INFO:teuthology.orchestra.run.smithi057.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-25T23:36:12.281 INFO:teuthology.orchestra.run.smithi057.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T23:36:12.377 INFO:teuthology.orchestra.run.smithi057.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-25T23:36:12.451 INFO:teuthology.orchestra.run.smithi057.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-25T23:36:13.591 INFO:teuthology.orchestra.run.smithi057.stdout:Reading package lists... 2024-04-25T23:36:13.608 DEBUG:teuthology.orchestra.run.smithi057:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-25T23:36:13.669 INFO:teuthology.orchestra.run.smithi057.stdout:Reading package lists... 2024-04-25T23:36:13.868 INFO:teuthology.orchestra.run.smithi057.stdout:Building dependency tree... 2024-04-25T23:36:13.868 INFO:teuthology.orchestra.run.smithi057.stdout:Reading state information... 2024-04-25T23:36:14.054 INFO:teuthology.orchestra.run.smithi057.stdout:linux-image-generic is already the newest version (5.15.0.105.102). 2024-04-25T23:36:14.054 INFO:teuthology.orchestra.run.smithi057.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T23:36:14.054 INFO:teuthology.orchestra.run.smithi057.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T23:36:14.054 INFO:teuthology.orchestra.run.smithi057.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T23:36:14.054 INFO:teuthology.orchestra.run.smithi057.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T23:36:14.055 INFO:teuthology.orchestra.run.smithi057.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T23:36:14.055 INFO:teuthology.orchestra.run.smithi057.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T23:36:14.055 INFO:teuthology.orchestra.run.smithi057.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T23:36:14.098 INFO:teuthology.orchestra.run.smithi057.stdout:0 upgraded, 0 newly installed, 0 to remove and 315 not upgraded. 2024-04-25T23:36:14.100 DEBUG:teuthology.orchestra.run.smithi057:> dpkg -s linux-image-generic 2024-04-25T23:36:14.116 INFO:teuthology.orchestra.run.smithi057.stdout:Package: linux-image-generic 2024-04-25T23:36:14.116 INFO:teuthology.orchestra.run.smithi057.stdout:Status: install ok installed 2024-04-25T23:36:14.116 INFO:teuthology.orchestra.run.smithi057.stdout:Priority: optional 2024-04-25T23:36:14.116 INFO:teuthology.orchestra.run.smithi057.stdout:Section: kernel 2024-04-25T23:36:14.116 INFO:teuthology.orchestra.run.smithi057.stdout:Installed-Size: 21 2024-04-25T23:36:14.116 INFO:teuthology.orchestra.run.smithi057.stdout:Maintainer: Ubuntu Kernel Team 2024-04-25T23:36:14.117 INFO:teuthology.orchestra.run.smithi057.stdout:Architecture: amd64 2024-04-25T23:36:14.117 INFO:teuthology.orchestra.run.smithi057.stdout:Source: linux-meta 2024-04-25T23:36:14.117 INFO:teuthology.orchestra.run.smithi057.stdout:Version: 5.15.0.105.102 2024-04-25T23:36:14.117 INFO:teuthology.orchestra.run.smithi057.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-25T23:36:14.117 INFO:teuthology.orchestra.run.smithi057.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-25T23:36:14.117 INFO:teuthology.orchestra.run.smithi057.stdout:Recommends: thermald 2024-04-25T23:36:14.117 INFO:teuthology.orchestra.run.smithi057.stdout:Description: Generic Linux kernel image 2024-04-25T23:36:14.117 INFO:teuthology.orchestra.run.smithi057.stdout: This package will always depend on the latest generic kernel image 2024-04-25T23:36:14.117 INFO:teuthology.orchestra.run.smithi057.stdout: available. 2024-04-25T23:36:14.118 DEBUG:teuthology.orchestra.run.smithi057:> mktemp 2024-04-25T23:36:14.164 INFO:teuthology.orchestra.run.smithi057.stdout:/tmp/tmp.Q8H99Q4o5q 2024-04-25T23:36:14.165 DEBUG:teuthology.orchestra.run.smithi057:> sudo cp /boot/grub/grub.cfg /tmp/tmp.Q8H99Q4o5q 2024-04-25T23:36:14.221 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod 0666 /tmp/tmp.Q8H99Q4o5q 2024-04-25T23:36:14.352 DEBUG:teuthology.orchestra.remote:smithi057:/tmp/tmp.Q8H99Q4o5q is 10KB 2024-04-25T23:36:14.365 DEBUG:teuthology.orchestra.run.smithi057:> rm -fr /tmp/tmp.Q8H99Q4o5q 2024-04-25T23:36:14.370 DEBUG:teuthology.orchestra.run.smithi057:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-04-25T23:36:14.424 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-04-25T23:36:14.424 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-04-25T23:36:14.424 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-04-25T23:36:14.527 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-105-generic 2024-04-25T23:36:14.527 DEBUG:teuthology.orchestra.run.smithi057:> sudo update-grub 2024-04-25T23:36:15.641 INFO:teuthology.orchestra.run.smithi057.stderr:Sourcing file `/etc/default/grub' 2024-04-25T23:36:15.642 INFO:teuthology.orchestra.run.smithi057.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T23:36:15.644 INFO:teuthology.orchestra.run.smithi057.stderr:Generating grub configuration file ... 2024-04-25T23:36:15.822 INFO:teuthology.orchestra.run.smithi057.stderr:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-25T23:36:15.828 INFO:teuthology.orchestra.run.smithi057.stderr:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-25T23:36:16.089 INFO:teuthology.orchestra.run.smithi057.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T23:36:16.094 INFO:teuthology.orchestra.run.smithi057.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T23:36:16.136 INFO:teuthology.orchestra.run.smithi057.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T23:36:16.141 INFO:teuthology.orchestra.run.smithi057.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T23:36:16.260 INFO:teuthology.orchestra.run.smithi057.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T23:36:16.260 INFO:teuthology.orchestra.run.smithi057.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T23:36:16.260 INFO:teuthology.orchestra.run.smithi057.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T23:36:16.288 INFO:teuthology.orchestra.run.smithi057.stderr:done 2024-04-25T23:36:16.291 DEBUG:teuthology.orchestra.run.smithi057:> sudo shutdown -r now 2024-04-25T23:36:22.516 INFO:teuthology.orchestra.run.smithi080.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-04-25T23:36:22.516 INFO:teuthology.orchestra.run.smithi080.stdout:Sourcing file `/etc/default/grub' 2024-04-25T23:36:22.540 INFO:teuthology.orchestra.run.smithi080.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T23:36:22.559 INFO:teuthology.orchestra.run.smithi080.stdout:Generating grub configuration file ... 2024-04-25T23:36:22.845 INFO:teuthology.orchestra.run.smithi080.stdout:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-25T23:36:22.866 INFO:teuthology.orchestra.run.smithi080.stdout:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-25T23:36:23.151 INFO:teuthology.orchestra.run.smithi080.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T23:36:23.157 INFO:teuthology.orchestra.run.smithi080.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T23:36:23.233 INFO:teuthology.orchestra.run.smithi080.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T23:36:23.239 INFO:teuthology.orchestra.run.smithi080.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T23:36:23.420 INFO:teuthology.orchestra.run.smithi080.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T23:36:23.420 INFO:teuthology.orchestra.run.smithi080.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T23:36:23.420 INFO:teuthology.orchestra.run.smithi080.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T23:36:23.467 INFO:teuthology.orchestra.run.smithi080.stdout:done 2024-04-25T23:36:23.907 DEBUG:teuthology.orchestra.run.smithi080:> dpkg -s linux-image-generic 2024-04-25T23:36:23.928 INFO:teuthology.orchestra.run.smithi080.stdout:Package: linux-image-generic 2024-04-25T23:36:23.929 INFO:teuthology.orchestra.run.smithi080.stdout:Status: install ok installed 2024-04-25T23:36:23.929 INFO:teuthology.orchestra.run.smithi080.stdout:Priority: optional 2024-04-25T23:36:23.929 INFO:teuthology.orchestra.run.smithi080.stdout:Section: kernel 2024-04-25T23:36:23.929 INFO:teuthology.orchestra.run.smithi080.stdout:Installed-Size: 21 2024-04-25T23:36:23.929 INFO:teuthology.orchestra.run.smithi080.stdout:Maintainer: Ubuntu Kernel Team 2024-04-25T23:36:23.929 INFO:teuthology.orchestra.run.smithi080.stdout:Architecture: amd64 2024-04-25T23:36:23.929 INFO:teuthology.orchestra.run.smithi080.stdout:Source: linux-meta 2024-04-25T23:36:23.929 INFO:teuthology.orchestra.run.smithi080.stdout:Version: 5.15.0.105.102 2024-04-25T23:36:23.929 INFO:teuthology.orchestra.run.smithi080.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-25T23:36:23.929 INFO:teuthology.orchestra.run.smithi080.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-25T23:36:23.929 INFO:teuthology.orchestra.run.smithi080.stdout:Recommends: thermald 2024-04-25T23:36:23.929 INFO:teuthology.orchestra.run.smithi080.stdout:Description: Generic Linux kernel image 2024-04-25T23:36:23.929 INFO:teuthology.orchestra.run.smithi080.stdout: This package will always depend on the latest generic kernel image 2024-04-25T23:36:23.930 INFO:teuthology.orchestra.run.smithi080.stdout: available. 2024-04-25T23:36:23.930 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-105-generic 2024-04-25T23:36:23.930 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-25T23:36:23.930 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-25T23:36:23.930 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-04-25T23:36:23.930 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi080.front.sepia.ceph.com, path=None, version=distro) 2024-04-25T23:36:23.930 DEBUG:teuthology.orchestra.run.smithi080:> sudo apt-get clean 2024-04-25T23:36:24.062 DEBUG:teuthology.orchestra.run.smithi080:> sudo apt-get update 2024-04-25T23:36:24.255 INFO:teuthology.orchestra.run.smithi080.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T23:36:24.288 INFO:teuthology.orchestra.run.smithi080.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-25T23:36:24.292 INFO:teuthology.orchestra.run.smithi080.stdout:Hit:3 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-25T23:36:24.330 INFO:teuthology.orchestra.run.smithi080.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-25T23:36:25.624 INFO:teuthology.orchestra.run.smithi080.stdout:Reading package lists... 2024-04-25T23:36:25.642 DEBUG:teuthology.orchestra.run.smithi080:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-25T23:36:25.693 INFO:teuthology.orchestra.run.smithi080.stdout:Reading package lists... 2024-04-25T23:36:25.870 INFO:teuthology.orchestra.run.smithi080.stdout:Building dependency tree... 2024-04-25T23:36:25.870 INFO:teuthology.orchestra.run.smithi080.stdout:Reading state information... 2024-04-25T23:36:26.024 INFO:teuthology.orchestra.run.smithi080.stdout:linux-image-generic is already the newest version (5.15.0.105.102). 2024-04-25T23:36:26.024 INFO:teuthology.orchestra.run.smithi080.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T23:36:26.024 INFO:teuthology.orchestra.run.smithi080.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T23:36:26.024 INFO:teuthology.orchestra.run.smithi080.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T23:36:26.025 INFO:teuthology.orchestra.run.smithi080.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T23:36:26.025 INFO:teuthology.orchestra.run.smithi080.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T23:36:26.025 INFO:teuthology.orchestra.run.smithi080.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T23:36:26.025 INFO:teuthology.orchestra.run.smithi080.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T23:36:26.062 INFO:teuthology.orchestra.run.smithi080.stdout:0 upgraded, 0 newly installed, 0 to remove and 315 not upgraded. 2024-04-25T23:36:26.064 DEBUG:teuthology.orchestra.run.smithi080:> dpkg -s linux-image-generic 2024-04-25T23:36:26.117 INFO:teuthology.orchestra.run.smithi080.stdout:Package: linux-image-generic 2024-04-25T23:36:26.117 INFO:teuthology.orchestra.run.smithi080.stdout:Status: install ok installed 2024-04-25T23:36:26.117 INFO:teuthology.orchestra.run.smithi080.stdout:Priority: optional 2024-04-25T23:36:26.117 INFO:teuthology.orchestra.run.smithi080.stdout:Section: kernel 2024-04-25T23:36:26.117 INFO:teuthology.orchestra.run.smithi080.stdout:Installed-Size: 21 2024-04-25T23:36:26.117 INFO:teuthology.orchestra.run.smithi080.stdout:Maintainer: Ubuntu Kernel Team 2024-04-25T23:36:26.117 INFO:teuthology.orchestra.run.smithi080.stdout:Architecture: amd64 2024-04-25T23:36:26.117 INFO:teuthology.orchestra.run.smithi080.stdout:Source: linux-meta 2024-04-25T23:36:26.117 INFO:teuthology.orchestra.run.smithi080.stdout:Version: 5.15.0.105.102 2024-04-25T23:36:26.117 INFO:teuthology.orchestra.run.smithi080.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-25T23:36:26.117 INFO:teuthology.orchestra.run.smithi080.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-25T23:36:26.117 INFO:teuthology.orchestra.run.smithi080.stdout:Recommends: thermald 2024-04-25T23:36:26.117 INFO:teuthology.orchestra.run.smithi080.stdout:Description: Generic Linux kernel image 2024-04-25T23:36:26.118 INFO:teuthology.orchestra.run.smithi080.stdout: This package will always depend on the latest generic kernel image 2024-04-25T23:36:26.118 INFO:teuthology.orchestra.run.smithi080.stdout: available. 2024-04-25T23:36:26.118 DEBUG:teuthology.orchestra.run.smithi080:> mktemp 2024-04-25T23:36:26.165 INFO:teuthology.orchestra.run.smithi080.stdout:/tmp/tmp.C8mcYxu1aY 2024-04-25T23:36:26.165 DEBUG:teuthology.orchestra.run.smithi080:> sudo cp /boot/grub/grub.cfg /tmp/tmp.C8mcYxu1aY 2024-04-25T23:36:26.220 DEBUG:teuthology.orchestra.run.smithi080:> sudo chmod 0666 /tmp/tmp.C8mcYxu1aY 2024-04-25T23:36:26.345 DEBUG:teuthology.orchestra.remote:smithi080:/tmp/tmp.C8mcYxu1aY is 10KB 2024-04-25T23:36:26.358 DEBUG:teuthology.orchestra.run.smithi080:> rm -fr /tmp/tmp.C8mcYxu1aY 2024-04-25T23:36:26.363 DEBUG:teuthology.orchestra.run.smithi080:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-04-25T23:36:26.416 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T23:36:26.417 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-04-25T23:36:26.417 DEBUG:teuthology.orchestra.run.smithi080:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-04-25T23:36:26.484 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-105-generic 2024-04-25T23:36:26.484 DEBUG:teuthology.orchestra.run.smithi080:> sudo update-grub 2024-04-25T23:36:27.687 INFO:teuthology.orchestra.run.smithi080.stderr:Sourcing file `/etc/default/grub' 2024-04-25T23:36:27.688 INFO:teuthology.orchestra.run.smithi080.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T23:36:27.690 INFO:teuthology.orchestra.run.smithi080.stderr:Generating grub configuration file ... 2024-04-25T23:36:27.870 INFO:teuthology.orchestra.run.smithi080.stderr:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-25T23:36:27.876 INFO:teuthology.orchestra.run.smithi080.stderr:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-25T23:36:28.120 INFO:teuthology.orchestra.run.smithi080.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T23:36:28.126 INFO:teuthology.orchestra.run.smithi080.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T23:36:28.167 INFO:teuthology.orchestra.run.smithi080.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T23:36:28.171 INFO:teuthology.orchestra.run.smithi080.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T23:36:28.314 INFO:teuthology.orchestra.run.smithi080.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T23:36:28.314 INFO:teuthology.orchestra.run.smithi080.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T23:36:28.314 INFO:teuthology.orchestra.run.smithi080.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T23:36:28.326 INFO:teuthology.orchestra.run.smithi080.stderr:done 2024-04-25T23:36:28.339 DEBUG:teuthology.orchestra.run.smithi080:> sudo shutdown -r now 2024-04-25T23:36:46.310 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-04-25T23:36:46.311 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi057.front.sepia.ceph.com' 2024-04-25T23:36:46.312 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi057.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T23:36:58.349 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-04-25T23:36:58.350 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi080.front.sepia.ceph.com' 2024-04-25T23:36:58.350 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T23:37:20.927 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.57 2024-04-25T23:37:29.934 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi057.front.sepia.ceph.com' 2024-04-25T23:37:29.935 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi057.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T23:37:32.958 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2024-04-25T23:37:41.966 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi080.front.sepia.ceph.com' 2024-04-25T23:37:41.967 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T23:37:48.318 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.57 2024-04-25T23:38:00.331 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi057.front.sepia.ceph.com' 2024-04-25T23:38:00.331 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi057.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T23:38:00.350 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2024-04-25T23:38:03.390 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.57 2024-04-25T23:38:12.360 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi080.front.sepia.ceph.com' 2024-04-25T23:38:12.361 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T23:38:12.754 ERROR:teuthology.orchestra.connection:Error authenticating with smithi080.front.sepia.ceph.com: SSHException('No existing session') (No SSH private key found!) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/connection.py", line 106, in connect ssh.connect(**connect_args) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 485, in connect self._auth( File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 818, in _auth raise saved_exception File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 794, in _auth self._transport.auth_publickey(username, key) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/virtualenv/lib/python3.8/site-packages/paramiko/transport.py", line 1664, in auth_publickey raise SSHException("No existing session") paramiko.ssh_exception.SSHException: No existing session 2024-04-25T23:38:16.837 DEBUG:teuthology.orchestra.run.smithi080:> true 2024-04-25T23:38:17.558 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi080.front.sepia.ceph.com' 2024-04-25T23:38:17.558 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-105-generic"... 2024-04-25T23:38:17.558 DEBUG:teuthology.orchestra.run.smithi080:> uname -r 2024-04-25T23:38:17.605 INFO:teuthology.orchestra.run.smithi080.stdout:5.15.0-105-generic 2024-04-25T23:38:17.605 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-105-generic vs 5.15.0-105-generic 2024-04-25T23:38:17.605 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-25T23:38:17.605 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-25T23:38:18.392 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi057.front.sepia.ceph.com' 2024-04-25T23:38:18.392 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi057.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T23:38:18.606 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-25T23:38:18.606 DEBUG:teuthology.orchestra.run.smithi080:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-25T23:38:18.623 DEBUG:teuthology.orchestra.run.smithi057:> true 2024-04-25T23:38:18.720 INFO:teuthology.orchestra.run.smithi080.stdout:ttyS1 2024-04-25T23:38:18.754 DEBUG:teuthology.parallel:result is None 2024-04-25T23:38:19.123 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi057.front.sepia.ceph.com' 2024-04-25T23:38:19.123 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-105-generic"... 2024-04-25T23:38:19.123 DEBUG:teuthology.orchestra.run.smithi057:> uname -r 2024-04-25T23:38:19.170 INFO:teuthology.orchestra.run.smithi057.stdout:5.15.0-105-generic 2024-04-25T23:38:19.170 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-105-generic vs 5.15.0-105-generic 2024-04-25T23:38:19.170 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-25T23:38:19.170 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-25T23:38:20.171 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-25T23:38:20.171 DEBUG:teuthology.orchestra.run.smithi057:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-25T23:38:20.250 INFO:teuthology.orchestra.run.smithi057.stdout:ttyS1 2024-04-25T23:38:20.284 DEBUG:teuthology.parallel:result is None 2024-04-25T23:38:20.284 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T23:38:20.291 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T23:38:20.292 DEBUG:teuthology.orchestra.run.smithi057:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T23:38:20.294 DEBUG:teuthology.orchestra.run.smithi080:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T23:38:20.300 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T23:38:20.307 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T23:38:20.314 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T23:38:20.315 DEBUG:teuthology.orchestra.run.smithi057:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T23:38:20.340 DEBUG:teuthology.orchestra.run.smithi080:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T23:38:20.402 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T23:38:20.409 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T23:38:20.409 DEBUG:teuthology.orchestra.run.smithi057:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T23:38:20.412 DEBUG:teuthology.orchestra.run.smithi080:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T23:38:20.426 INFO:teuthology.orchestra.run.smithi057.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T23:38:20.434 INFO:teuthology.orchestra.run.smithi080.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T23:38:20.436 INFO:teuthology.orchestra.run.smithi057.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T23:38:20.442 INFO:teuthology.orchestra.run.smithi080.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T23:38:20.444 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T23:38:20.451 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T23:38:20.451 DEBUG:teuthology.orchestra.run.smithi057:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T23:38:20.480 DEBUG:teuthology.orchestra.run.smithi080:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T23:38:20.500 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T23:38:20.523 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T23:38:20.523 DEBUG:teuthology.orchestra.run.smithi057:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T23:38:20.536 DEBUG:teuthology.orchestra.run.smithi080:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T23:38:20.545 DEBUG:teuthology.orchestra.run.smithi057:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T23:38:20.591 DEBUG:teuthology.orchestra.run.smithi057:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T23:38:20.639 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-04-25T23:38:20.639 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T23:38:20.701 DEBUG:teuthology.orchestra.run.smithi080:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T23:38:20.707 DEBUG:teuthology.orchestra.run.smithi080:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T23:38:20.753 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T23:38:20.753 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T23:38:20.820 DEBUG:teuthology.orchestra.run.smithi057:> sudo service rsyslog restart 2024-04-25T23:38:20.823 DEBUG:teuthology.orchestra.run.smithi080:> sudo service rsyslog restart 2024-04-25T23:38:20.900 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T23:38:20.908 INFO:teuthology.task.internal:Starting timer... 2024-04-25T23:38:20.908 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T23:38:20.920 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T23:38:20.929 DEBUG:teuthology.task.selinux:Excluding smithi057: OS 'ubuntu' does not support SELinux 2024-04-25T23:38:20.929 DEBUG:teuthology.task.selinux:Excluding smithi080: OS 'ubuntu' does not support SELinux 2024-04-25T23:38:20.929 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T23:38:20.929 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-04-25T23:38:20.929 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T23:38:20.929 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T23:38:20.939 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T23:38:20.958 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T23:38:20.959 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi057.front.sepia.ceph.com,smithi080.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T23:45:54.353 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi057.front.sepia.ceph.com'), Remote(name='ubuntu@smithi080.front.sepia.ceph.com')] 2024-04-25T23:45:54.354 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi057.front.sepia.ceph.com' 2024-04-25T23:45:54.356 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi057.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T23:45:54.444 DEBUG:teuthology.orchestra.run.smithi057:> true 2024-04-25T23:45:54.522 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi057.front.sepia.ceph.com' 2024-04-25T23:45:54.522 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi080.front.sepia.ceph.com' 2024-04-25T23:45:54.523 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T23:45:54.604 DEBUG:teuthology.orchestra.run.smithi080:> true 2024-04-25T23:45:54.682 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi080.front.sepia.ceph.com' 2024-04-25T23:45:54.682 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T23:45:54.694 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T23:45:54.695 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T23:45:54.695 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T23:45:54.697 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T23:45:54.698 DEBUG:teuthology.orchestra.run.smithi080:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T23:45:54.727 INFO:teuthology.orchestra.run.smithi057.stdout:25 Apr 23:45:54 ntpd[17125]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-25T23:45:54.727 INFO:teuthology.orchestra.run.smithi057.stdout:25 Apr 23:45:54 ntpd[17125]: Command line: ntpd -gq 2024-04-25T23:45:54.727 INFO:teuthology.orchestra.run.smithi057.stdout:25 Apr 23:45:54 ntpd[17125]: ---------------------------------------------------- 2024-04-25T23:45:54.727 INFO:teuthology.orchestra.run.smithi057.stdout:25 Apr 23:45:54 ntpd[17125]: ntp-4 is maintained by Network Time Foundation, 2024-04-25T23:45:54.727 INFO:teuthology.orchestra.run.smithi057.stdout:25 Apr 23:45:54 ntpd[17125]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-25T23:45:54.727 INFO:teuthology.orchestra.run.smithi057.stdout:25 Apr 23:45:54 ntpd[17125]: corporation. Support and training for ntp-4 are 2024-04-25T23:45:54.727 INFO:teuthology.orchestra.run.smithi057.stdout:25 Apr 23:45:54 ntpd[17125]: available at https://www.nwtime.org/support 2024-04-25T23:45:54.727 INFO:teuthology.orchestra.run.smithi057.stdout:25 Apr 23:45:54 ntpd[17125]: ---------------------------------------------------- 2024-04-25T23:45:54.728 INFO:teuthology.orchestra.run.smithi057.stdout:25 Apr 23:45:54 ntpd[17125]: proto: precision = 0.069 usec (-24) 2024-04-25T23:45:54.728 INFO:teuthology.orchestra.run.smithi057.stdout:25 Apr 23:45:54 ntpd[17125]: basedate set to 2022-02-04 2024-04-25T23:45:54.728 INFO:teuthology.orchestra.run.smithi057.stdout:25 Apr 23:45:54 ntpd[17125]: gps base set to 2022-02-06 (week 2196) 2024-04-25T23:45:54.728 INFO:teuthology.orchestra.run.smithi057.stdout:25 Apr 23:45:54 ntpd[17125]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T23:45:54.729 INFO:teuthology.orchestra.run.smithi057.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T23:45:54.729 INFO:teuthology.orchestra.run.smithi057.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-25T23:45:54.730 INFO:teuthology.orchestra.run.smithi057.stdout:25 Apr 23:45:54 ntpd[17125]: restrict ::: KOD does nothing without LIMITED. 2024-04-25T23:45:54.730 INFO:teuthology.orchestra.run.smithi057.stdout:25 Apr 23:45:54 ntpd[17125]: Listen and drop on 0 v6wildcard [::]:123 2024-04-25T23:45:54.730 INFO:teuthology.orchestra.run.smithi057.stdout:25 Apr 23:45:54 ntpd[17125]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-25T23:45:54.730 INFO:teuthology.orchestra.run.smithi057.stdout:25 Apr 23:45:54 ntpd[17125]: Listen normally on 2 lo 127.0.0.1:123 2024-04-25T23:45:54.730 INFO:teuthology.orchestra.run.smithi057.stdout:25 Apr 23:45:54 ntpd[17125]: Listen normally on 3 ens1f0 172.21.15.57:123 2024-04-25T23:45:54.730 INFO:teuthology.orchestra.run.smithi057.stdout:25 Apr 23:45:54 ntpd[17125]: Listen normally on 4 lo [::1]:123 2024-04-25T23:45:54.730 INFO:teuthology.orchestra.run.smithi057.stdout:25 Apr 23:45:54 ntpd[17125]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:febd:1562%4]:123 2024-04-25T23:45:54.730 INFO:teuthology.orchestra.run.smithi057.stdout:25 Apr 23:45:54 ntpd[17125]: Listening on routing socket on fd #22 for interface updates 2024-04-25T23:45:54.753 INFO:teuthology.orchestra.run.smithi080.stdout:25 Apr 23:45:54 ntpd[17229]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-25T23:45:54.753 INFO:teuthology.orchestra.run.smithi080.stdout:25 Apr 23:45:54 ntpd[17229]: Command line: ntpd -gq 2024-04-25T23:45:54.753 INFO:teuthology.orchestra.run.smithi080.stdout:25 Apr 23:45:54 ntpd[17229]: ---------------------------------------------------- 2024-04-25T23:45:54.753 INFO:teuthology.orchestra.run.smithi080.stdout:25 Apr 23:45:54 ntpd[17229]: ntp-4 is maintained by Network Time Foundation, 2024-04-25T23:45:54.753 INFO:teuthology.orchestra.run.smithi080.stdout:25 Apr 23:45:54 ntpd[17229]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-25T23:45:54.753 INFO:teuthology.orchestra.run.smithi080.stdout:25 Apr 23:45:54 ntpd[17229]: corporation. Support and training for ntp-4 are 2024-04-25T23:45:54.754 INFO:teuthology.orchestra.run.smithi080.stdout:25 Apr 23:45:54 ntpd[17229]: available at https://www.nwtime.org/support 2024-04-25T23:45:54.754 INFO:teuthology.orchestra.run.smithi080.stdout:25 Apr 23:45:54 ntpd[17229]: ---------------------------------------------------- 2024-04-25T23:45:54.754 INFO:teuthology.orchestra.run.smithi080.stdout:25 Apr 23:45:54 ntpd[17229]: proto: precision = 0.061 usec (-24) 2024-04-25T23:45:54.754 INFO:teuthology.orchestra.run.smithi080.stdout:25 Apr 23:45:54 ntpd[17229]: basedate set to 2022-02-04 2024-04-25T23:45:54.754 INFO:teuthology.orchestra.run.smithi080.stdout:25 Apr 23:45:54 ntpd[17229]: gps base set to 2022-02-06 (week 2196) 2024-04-25T23:45:54.755 INFO:teuthology.orchestra.run.smithi080.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T23:45:54.755 INFO:teuthology.orchestra.run.smithi080.stdout:25 Apr 23:45:54 ntpd[17229]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T23:45:54.755 INFO:teuthology.orchestra.run.smithi080.stdout:25 Apr 23:45:54 ntpd[17229]: restrict ::: KOD does nothing without LIMITED. 2024-04-25T23:45:54.755 INFO:teuthology.orchestra.run.smithi080.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-25T23:45:54.756 INFO:teuthology.orchestra.run.smithi080.stdout:25 Apr 23:45:54 ntpd[17229]: Listen and drop on 0 v6wildcard [::]:123 2024-04-25T23:45:54.756 INFO:teuthology.orchestra.run.smithi080.stdout:25 Apr 23:45:54 ntpd[17229]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-25T23:45:54.757 INFO:teuthology.orchestra.run.smithi080.stdout:25 Apr 23:45:54 ntpd[17229]: Listen normally on 2 lo 127.0.0.1:123 2024-04-25T23:45:54.757 INFO:teuthology.orchestra.run.smithi080.stdout:25 Apr 23:45:54 ntpd[17229]: Listen normally on 3 enp3s0f1 172.21.15.80:123 2024-04-25T23:45:54.757 INFO:teuthology.orchestra.run.smithi080.stdout:25 Apr 23:45:54 ntpd[17229]: Listen normally on 4 lo [::1]:123 2024-04-25T23:45:54.757 INFO:teuthology.orchestra.run.smithi080.stdout:25 Apr 23:45:54 ntpd[17229]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:cba9%5]:123 2024-04-25T23:45:54.757 INFO:teuthology.orchestra.run.smithi080.stdout:25 Apr 23:45:54 ntpd[17229]: Listening on routing socket on fd #22 for interface updates 2024-04-25T23:45:55.755 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 23:45:55 ntpd[17229]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T23:45:55.755 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 23:45:55 ntpd[17229]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T23:45:55.755 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 23:45:55 ntpd[17229]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T23:45:55.755 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 23:45:55 ntpd[17229]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T23:45:55.755 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 23:45:55 ntpd[17229]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T23:45:55.755 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 23:45:55 ntpd[17229]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T23:45:55.755 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 23:45:55 ntpd[17229]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T23:45:55.755 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 23:45:55 ntpd[17229]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T23:45:55.755 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 23:45:55 ntpd[17229]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T23:45:55.755 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 23:45:55 ntpd[17229]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T23:45:55.755 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 23:45:55 ntpd[17229]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T23:45:55.755 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 23:45:55 ntpd[17229]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T23:45:56.727 INFO:teuthology.orchestra.run.smithi057.stderr:25 Apr 23:45:56 ntpd[17125]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T23:45:56.727 INFO:teuthology.orchestra.run.smithi057.stderr:25 Apr 23:45:56 ntpd[17125]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T23:45:57.728 INFO:teuthology.orchestra.run.smithi057.stderr:25 Apr 23:45:57 ntpd[17125]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T23:45:57.728 INFO:teuthology.orchestra.run.smithi057.stderr:25 Apr 23:45:57 ntpd[17125]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T23:45:57.728 INFO:teuthology.orchestra.run.smithi057.stderr:25 Apr 23:45:57 ntpd[17125]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T23:45:57.728 INFO:teuthology.orchestra.run.smithi057.stderr:25 Apr 23:45:57 ntpd[17125]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T23:45:57.728 INFO:teuthology.orchestra.run.smithi057.stderr:25 Apr 23:45:57 ntpd[17125]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T23:45:57.728 INFO:teuthology.orchestra.run.smithi057.stderr:25 Apr 23:45:57 ntpd[17125]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T23:45:57.728 INFO:teuthology.orchestra.run.smithi057.stderr:25 Apr 23:45:57 ntpd[17125]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T23:45:57.728 INFO:teuthology.orchestra.run.smithi057.stderr:25 Apr 23:45:57 ntpd[17125]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T23:45:57.729 INFO:teuthology.orchestra.run.smithi057.stderr:25 Apr 23:45:57 ntpd[17125]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T23:45:57.729 INFO:teuthology.orchestra.run.smithi057.stderr:25 Apr 23:45:57 ntpd[17125]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T23:45:57.729 INFO:teuthology.orchestra.run.smithi057.stderr:25 Apr 23:45:57 ntpd[17125]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T23:45:57.729 INFO:teuthology.orchestra.run.smithi057.stderr:25 Apr 23:45:57 ntpd[17125]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T23:45:57.755 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 23:45:57 ntpd[17229]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T23:45:57.755 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 23:45:57 ntpd[17229]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T23:45:57.756 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 23:45:57 ntpd[17229]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T23:45:57.756 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 23:45:57 ntpd[17229]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T23:45:57.756 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 23:45:57 ntpd[17229]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T23:45:57.756 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 23:45:57 ntpd[17229]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T23:45:57.756 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 23:45:57 ntpd[17229]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T23:45:57.756 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 23:45:57 ntpd[17229]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T23:45:57.756 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 23:45:57 ntpd[17229]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T23:45:57.756 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 23:45:57 ntpd[17229]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T23:45:57.756 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 23:45:57 ntpd[17229]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T23:45:57.756 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 23:45:57 ntpd[17229]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T23:45:57.756 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 23:45:57 ntpd[17229]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T23:45:57.756 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 23:45:57 ntpd[17229]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T23:45:57.756 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 23:45:57 ntpd[17229]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T23:45:57.756 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 23:45:57 ntpd[17229]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T23:45:57.757 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 23:45:57 ntpd[17229]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T23:45:57.757 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 23:45:57 ntpd[17229]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T23:45:57.757 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 23:45:57 ntpd[17229]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T23:45:57.757 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 23:45:57 ntpd[17229]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T23:45:59.729 INFO:teuthology.orchestra.run.smithi057.stderr:25 Apr 23:45:59 ntpd[17125]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T23:45:59.729 INFO:teuthology.orchestra.run.smithi057.stderr:25 Apr 23:45:59 ntpd[17125]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T23:45:59.729 INFO:teuthology.orchestra.run.smithi057.stderr:25 Apr 23:45:59 ntpd[17125]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T23:45:59.729 INFO:teuthology.orchestra.run.smithi057.stderr:25 Apr 23:45:59 ntpd[17125]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T23:45:59.729 INFO:teuthology.orchestra.run.smithi057.stderr:25 Apr 23:45:59 ntpd[17125]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T23:45:59.729 INFO:teuthology.orchestra.run.smithi057.stderr:25 Apr 23:45:59 ntpd[17125]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T23:45:59.729 INFO:teuthology.orchestra.run.smithi057.stderr:25 Apr 23:45:59 ntpd[17125]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T23:45:59.729 INFO:teuthology.orchestra.run.smithi057.stderr:25 Apr 23:45:59 ntpd[17125]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T23:45:59.729 INFO:teuthology.orchestra.run.smithi057.stderr:25 Apr 23:45:59 ntpd[17125]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T23:45:59.729 INFO:teuthology.orchestra.run.smithi057.stderr:25 Apr 23:45:59 ntpd[17125]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T23:45:59.729 INFO:teuthology.orchestra.run.smithi057.stderr:25 Apr 23:45:59 ntpd[17125]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T23:45:59.729 INFO:teuthology.orchestra.run.smithi057.stderr:25 Apr 23:45:59 ntpd[17125]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T23:45:59.729 INFO:teuthology.orchestra.run.smithi057.stderr:25 Apr 23:45:59 ntpd[17125]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T23:45:59.730 INFO:teuthology.orchestra.run.smithi057.stderr:25 Apr 23:45:59 ntpd[17125]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T23:45:59.730 INFO:teuthology.orchestra.run.smithi057.stderr:25 Apr 23:45:59 ntpd[17125]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T23:45:59.730 INFO:teuthology.orchestra.run.smithi057.stderr:25 Apr 23:45:59 ntpd[17125]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T23:45:59.730 INFO:teuthology.orchestra.run.smithi057.stderr:25 Apr 23:45:59 ntpd[17125]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T23:45:59.730 INFO:teuthology.orchestra.run.smithi057.stderr:25 Apr 23:45:59 ntpd[17125]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T23:45:59.755 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 23:45:59 ntpd[17229]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T23:45:59.755 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 23:45:59 ntpd[17229]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T23:45:59.755 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 23:45:59 ntpd[17229]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T23:45:59.755 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 23:45:59 ntpd[17229]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T23:45:59.755 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 23:45:59 ntpd[17229]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T23:45:59.755 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 23:45:59 ntpd[17229]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T23:45:59.755 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 23:45:59 ntpd[17229]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T23:45:59.755 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 23:45:59 ntpd[17229]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T23:45:59.755 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 23:45:59 ntpd[17229]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T23:45:59.756 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 23:45:59 ntpd[17229]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T23:45:59.756 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 23:45:59 ntpd[17229]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T23:45:59.756 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 23:45:59 ntpd[17229]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T23:45:59.756 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 23:45:59 ntpd[17229]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T23:45:59.756 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 23:45:59 ntpd[17229]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T23:45:59.756 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 23:45:59 ntpd[17229]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T23:45:59.756 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 23:45:59 ntpd[17229]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T23:45:59.756 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 23:45:59 ntpd[17229]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T23:45:59.756 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 23:45:59 ntpd[17229]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T23:46:01.729 INFO:teuthology.orchestra.run.smithi057.stderr:25 Apr 23:46:01 ntpd[17125]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T23:46:01.729 INFO:teuthology.orchestra.run.smithi057.stderr:25 Apr 23:46:01 ntpd[17125]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T23:46:01.729 INFO:teuthology.orchestra.run.smithi057.stderr:25 Apr 23:46:01 ntpd[17125]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T23:46:01.729 INFO:teuthology.orchestra.run.smithi057.stderr:25 Apr 23:46:01 ntpd[17125]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T23:46:01.729 INFO:teuthology.orchestra.run.smithi057.stderr:25 Apr 23:46:01 ntpd[17125]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T23:46:01.729 INFO:teuthology.orchestra.run.smithi057.stderr:25 Apr 23:46:01 ntpd[17125]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T23:46:01.729 INFO:teuthology.orchestra.run.smithi057.stderr:25 Apr 23:46:01 ntpd[17125]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T23:46:01.729 INFO:teuthology.orchestra.run.smithi057.stderr:25 Apr 23:46:01 ntpd[17125]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T23:46:01.729 INFO:teuthology.orchestra.run.smithi057.stderr:25 Apr 23:46:01 ntpd[17125]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T23:46:01.729 INFO:teuthology.orchestra.run.smithi057.stderr:25 Apr 23:46:01 ntpd[17125]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T23:46:01.729 INFO:teuthology.orchestra.run.smithi057.stderr:25 Apr 23:46:01 ntpd[17125]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T23:46:01.730 INFO:teuthology.orchestra.run.smithi057.stderr:25 Apr 23:46:01 ntpd[17125]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T23:46:01.730 INFO:teuthology.orchestra.run.smithi057.stderr:25 Apr 23:46:01 ntpd[17125]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T23:46:01.730 INFO:teuthology.orchestra.run.smithi057.stderr:25 Apr 23:46:01 ntpd[17125]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T23:46:01.730 INFO:teuthology.orchestra.run.smithi057.stderr:25 Apr 23:46:01 ntpd[17125]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T23:46:01.730 INFO:teuthology.orchestra.run.smithi057.stderr:25 Apr 23:46:01 ntpd[17125]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T23:46:01.730 INFO:teuthology.orchestra.run.smithi057.stderr:25 Apr 23:46:01 ntpd[17125]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T23:46:01.730 INFO:teuthology.orchestra.run.smithi057.stderr:25 Apr 23:46:01 ntpd[17125]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T23:46:01.754 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 23:46:01 ntpd[17229]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T23:46:01.755 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 23:46:01 ntpd[17229]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T23:46:01.755 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 23:46:01 ntpd[17229]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T23:46:01.755 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 23:46:01 ntpd[17229]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T23:46:01.755 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 23:46:01 ntpd[17229]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T23:46:01.755 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 23:46:01 ntpd[17229]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T23:46:01.755 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 23:46:01 ntpd[17229]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T23:46:01.755 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 23:46:01 ntpd[17229]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T23:46:01.755 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 23:46:01 ntpd[17229]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T23:46:01.755 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 23:46:01 ntpd[17229]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T23:46:01.756 INFO:teuthology.orchestra.run.smithi080.stdout:25 Apr 23:46:01 ntpd[17229]: ntpd: time slew -0.000981 s 2024-04-25T23:46:01.756 INFO:teuthology.orchestra.run.smithi080.stdout:ntpd: time slew -0.000981s 2024-04-25T23:46:01.756 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 23:46:01 ntpd[17229]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-25T23:46:01.756 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 23:46:01 ntpd[17229]: can't open /var/log/ntpstats/loopstats.20240425: Permission denied 2024-04-25T23:46:01.871 INFO:teuthology.orchestra.run.smithi080.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T23:46:01.871 INFO:teuthology.orchestra.run.smithi080.stdout:============================================================================== 2024-04-25T23:46:01.871 INFO:teuthology.orchestra.run.smithi080.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T23:46:01.871 INFO:teuthology.orchestra.run.smithi080.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T23:46:01.871 INFO:teuthology.orchestra.run.smithi080.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T23:46:01.871 INFO:teuthology.orchestra.run.smithi080.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T23:46:03.729 INFO:teuthology.orchestra.run.smithi057.stderr:25 Apr 23:46:03 ntpd[17125]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T23:46:03.729 INFO:teuthology.orchestra.run.smithi057.stderr:25 Apr 23:46:03 ntpd[17125]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T23:46:03.729 INFO:teuthology.orchestra.run.smithi057.stderr:25 Apr 23:46:03 ntpd[17125]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T23:46:03.729 INFO:teuthology.orchestra.run.smithi057.stderr:25 Apr 23:46:03 ntpd[17125]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T23:46:03.729 INFO:teuthology.orchestra.run.smithi057.stderr:25 Apr 23:46:03 ntpd[17125]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T23:46:03.729 INFO:teuthology.orchestra.run.smithi057.stderr:25 Apr 23:46:03 ntpd[17125]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T23:46:03.730 INFO:teuthology.orchestra.run.smithi057.stderr:25 Apr 23:46:03 ntpd[17125]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T23:46:03.730 INFO:teuthology.orchestra.run.smithi057.stderr:25 Apr 23:46:03 ntpd[17125]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T23:46:03.730 INFO:teuthology.orchestra.run.smithi057.stderr:25 Apr 23:46:03 ntpd[17125]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T23:46:03.730 INFO:teuthology.orchestra.run.smithi057.stderr:25 Apr 23:46:03 ntpd[17125]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T23:46:03.730 INFO:teuthology.orchestra.run.smithi057.stderr:25 Apr 23:46:03 ntpd[17125]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T23:46:03.730 INFO:teuthology.orchestra.run.smithi057.stderr:25 Apr 23:46:03 ntpd[17125]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T23:46:03.730 INFO:teuthology.orchestra.run.smithi057.stderr:25 Apr 23:46:03 ntpd[17125]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T23:46:03.730 INFO:teuthology.orchestra.run.smithi057.stderr:25 Apr 23:46:03 ntpd[17125]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T23:46:03.730 INFO:teuthology.orchestra.run.smithi057.stdout:25 Apr 23:46:03 ntpd[17125]: ntpd: time slew -0.002578 s 2024-04-25T23:46:03.730 INFO:teuthology.orchestra.run.smithi057.stdout:ntpd: time slew -0.002578s 2024-04-25T23:46:03.731 INFO:teuthology.orchestra.run.smithi057.stderr:25 Apr 23:46:03 ntpd[17125]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-25T23:46:03.731 INFO:teuthology.orchestra.run.smithi057.stderr:25 Apr 23:46:03 ntpd[17125]: can't open /var/log/ntpstats/loopstats.20240425: Permission denied 2024-04-25T23:46:03.781 INFO:teuthology.orchestra.run.smithi057.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T23:46:03.781 INFO:teuthology.orchestra.run.smithi057.stdout:============================================================================== 2024-04-25T23:46:03.781 INFO:teuthology.orchestra.run.smithi057.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T23:46:03.781 INFO:teuthology.orchestra.run.smithi057.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T23:46:03.781 INFO:teuthology.orchestra.run.smithi057.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T23:46:03.781 INFO:teuthology.orchestra.run.smithi057.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T23:46:03.782 INFO:teuthology.run_tasks:Running task install... 2024-04-25T23:46:03.791 DEBUG:teuthology.task.install:project ceph 2024-04-25T23:46:03.791 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}} 2024-04-25T23:46:03.792 DEBUG:teuthology.task.install:config {'extra_system_packages': {'deb': ['sqlite3'], 'rpm': ['sqlite-devel']}, 'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'} 2024-04-25T23:46:03.792 INFO:teuthology.task.install:Using flavor: default 2024-04-25T23:46:03.798 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T23:46:03.798 INFO:teuthology.task.install:extra packages: [] 2024-04-25T23:46:03.799 DEBUG:teuthology.orchestra.run.smithi057:> sudo apt-key list | grep Ceph 2024-04-25T23:46:03.799 DEBUG:teuthology.orchestra.run.smithi080:> sudo apt-key list | grep Ceph 2024-04-25T23:46:03.860 INFO:teuthology.orchestra.run.smithi080.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-25T23:46:03.879 INFO:teuthology.orchestra.run.smithi057.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-25T23:46:03.887 INFO:teuthology.orchestra.run.smithi080.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-25T23:46:03.887 INFO:teuthology.orchestra.run.smithi080.stdout:uid [ unknown] Ceph.com (release key) 2024-04-25T23:46:03.888 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-04-25T23:46:03.888 INFO:teuthology.task.install.deb:Installing system (non-project) packages: sqlite3 on remote deb x86_64 2024-04-25T23:46:03.888 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T23:46:03.908 INFO:teuthology.orchestra.run.smithi057.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-25T23:46:03.908 INFO:teuthology.orchestra.run.smithi057.stdout:uid [ unknown] Ceph.com (release key) 2024-04-25T23:46:03.909 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-04-25T23:46:03.909 INFO:teuthology.task.install.deb:Installing system (non-project) packages: sqlite3 on remote deb x86_64 2024-04-25T23:46:03.909 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T23:46:04.107 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default/ 2024-04-25T23:46:04.108 INFO:teuthology.task.install.deb:Package version is 19.0.0-2445-gd2c45d97-1jammy 2024-04-25T23:46:04.135 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default/ 2024-04-25T23:46:04.136 INFO:teuthology.task.install.deb:Package version is 19.0.0-2445-gd2c45d97-1jammy 2024-04-25T23:46:04.247 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-04-25T23:46:04.248 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-25T23:46:04.262 DEBUG:teuthology.orchestra.run.smithi057:> sudo apt-get update 2024-04-25T23:46:04.285 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T23:46:04.285 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-25T23:46:04.299 DEBUG:teuthology.orchestra.run.smithi080:> sudo apt-get update 2024-04-25T23:46:04.441 INFO:teuthology.orchestra.run.smithi057.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T23:46:04.453 INFO:teuthology.orchestra.run.smithi057.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-25T23:46:04.455 INFO:teuthology.orchestra.run.smithi057.stdout:Hit:3 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-25T23:46:04.474 INFO:teuthology.orchestra.run.smithi057.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-25T23:46:04.488 INFO:teuthology.orchestra.run.smithi057.stdout:Ign:5 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy InRelease 2024-04-25T23:46:04.509 INFO:teuthology.orchestra.run.smithi057.stdout:Get:6 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-25T23:46:04.531 INFO:teuthology.orchestra.run.smithi057.stdout:Ign:7 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-25T23:46:04.552 INFO:teuthology.orchestra.run.smithi057.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,416 B] 2024-04-25T23:46:04.579 INFO:teuthology.orchestra.run.smithi057.stdout:Get:9 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.9 kB] 2024-04-25T23:46:04.583 INFO:teuthology.orchestra.run.smithi080.stdout:Ign:1 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy InRelease 2024-04-25T23:46:04.599 INFO:teuthology.orchestra.run.smithi080.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T23:46:04.599 INFO:teuthology.orchestra.run.smithi080.stdout:Hit:3 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-25T23:46:04.620 INFO:teuthology.orchestra.run.smithi080.stdout:Get:4 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-25T23:46:04.658 INFO:teuthology.orchestra.run.smithi080.stdout:Ign:5 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-25T23:46:04.672 INFO:teuthology.orchestra.run.smithi080.stdout:Hit:6 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-25T23:46:04.694 INFO:teuthology.orchestra.run.smithi080.stdout:Get:7 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.9 kB] 2024-04-25T23:46:04.754 INFO:teuthology.orchestra.run.smithi080.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-25T23:46:04.772 INFO:teuthology.orchestra.run.smithi080.stdout:Get:9 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,416 B] 2024-04-25T23:46:04.821 INFO:teuthology.orchestra.run.smithi057.stdout:Fetched 28.9 kB in 0s (77.6 kB/s) 2024-04-25T23:46:04.966 INFO:teuthology.orchestra.run.smithi080.stdout:Fetched 28.9 kB in 0s (59.8 kB/s) 2024-04-25T23:46:05.803 INFO:teuthology.orchestra.run.smithi057.stdout:Reading package lists... 2024-04-25T23:46:05.818 DEBUG:teuthology.orchestra.run.smithi057:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-2445-gd2c45d97-1jammy cephadm=19.0.0-2445-gd2c45d97-1jammy ceph-mds=19.0.0-2445-gd2c45d97-1jammy ceph-mgr=19.0.0-2445-gd2c45d97-1jammy ceph-common=19.0.0-2445-gd2c45d97-1jammy ceph-fuse=19.0.0-2445-gd2c45d97-1jammy ceph-test=19.0.0-2445-gd2c45d97-1jammy ceph-volume=19.0.0-2445-gd2c45d97-1jammy radosgw=19.0.0-2445-gd2c45d97-1jammy python3-rados=19.0.0-2445-gd2c45d97-1jammy python3-rgw=19.0.0-2445-gd2c45d97-1jammy python3-cephfs=19.0.0-2445-gd2c45d97-1jammy python3-rbd=19.0.0-2445-gd2c45d97-1jammy libcephfs2=19.0.0-2445-gd2c45d97-1jammy libcephfs-dev=19.0.0-2445-gd2c45d97-1jammy librados2=19.0.0-2445-gd2c45d97-1jammy librbd1=19.0.0-2445-gd2c45d97-1jammy rbd-fuse=19.0.0-2445-gd2c45d97-1jammy 2024-04-25T23:46:05.880 INFO:teuthology.orchestra.run.smithi057.stdout:Reading package lists... 2024-04-25T23:46:05.897 INFO:teuthology.orchestra.run.smithi080.stdout:Reading package lists... 2024-04-25T23:46:05.913 DEBUG:teuthology.orchestra.run.smithi080:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-2445-gd2c45d97-1jammy cephadm=19.0.0-2445-gd2c45d97-1jammy ceph-mds=19.0.0-2445-gd2c45d97-1jammy ceph-mgr=19.0.0-2445-gd2c45d97-1jammy ceph-common=19.0.0-2445-gd2c45d97-1jammy ceph-fuse=19.0.0-2445-gd2c45d97-1jammy ceph-test=19.0.0-2445-gd2c45d97-1jammy ceph-volume=19.0.0-2445-gd2c45d97-1jammy radosgw=19.0.0-2445-gd2c45d97-1jammy python3-rados=19.0.0-2445-gd2c45d97-1jammy python3-rgw=19.0.0-2445-gd2c45d97-1jammy python3-cephfs=19.0.0-2445-gd2c45d97-1jammy python3-rbd=19.0.0-2445-gd2c45d97-1jammy libcephfs2=19.0.0-2445-gd2c45d97-1jammy libcephfs-dev=19.0.0-2445-gd2c45d97-1jammy librados2=19.0.0-2445-gd2c45d97-1jammy librbd1=19.0.0-2445-gd2c45d97-1jammy rbd-fuse=19.0.0-2445-gd2c45d97-1jammy 2024-04-25T23:46:05.965 INFO:teuthology.orchestra.run.smithi080.stdout:Reading package lists... 2024-04-25T23:46:06.084 INFO:teuthology.orchestra.run.smithi057.stdout:Building dependency tree... 2024-04-25T23:46:06.084 INFO:teuthology.orchestra.run.smithi057.stdout:Reading state information... 2024-04-25T23:46:06.142 INFO:teuthology.orchestra.run.smithi080.stdout:Building dependency tree... 2024-04-25T23:46:06.142 INFO:teuthology.orchestra.run.smithi080.stdout:Reading state information... 2024-04-25T23:46:06.287 INFO:teuthology.orchestra.run.smithi057.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T23:46:06.288 INFO:teuthology.orchestra.run.smithi057.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T23:46:06.288 INFO:teuthology.orchestra.run.smithi057.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T23:46:06.288 INFO:teuthology.orchestra.run.smithi057.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T23:46:06.289 INFO:teuthology.orchestra.run.smithi057.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T23:46:06.289 INFO:teuthology.orchestra.run.smithi057.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T23:46:06.289 INFO:teuthology.orchestra.run.smithi057.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T23:46:06.289 INFO:teuthology.orchestra.run.smithi057.stdout:The following additional packages will be installed: 2024-04-25T23:46:06.289 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-25T23:46:06.292 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-25T23:46:06.292 INFO:teuthology.orchestra.run.smithi057.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T23:46:06.292 INFO:teuthology.orchestra.run.smithi057.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-25T23:46:06.292 INFO:teuthology.orchestra.run.smithi057.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-04-25T23:46:06.292 INFO:teuthology.orchestra.run.smithi057.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-04-25T23:46:06.292 INFO:teuthology.orchestra.run.smithi057.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-25T23:46:06.292 INFO:teuthology.orchestra.run.smithi057.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-25T23:46:06.292 INFO:teuthology.orchestra.run.smithi057.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-25T23:46:06.292 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-04-25T23:46:06.293 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-25T23:46:06.293 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-25T23:46:06.293 INFO:teuthology.orchestra.run.smithi057.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-25T23:46:06.293 INFO:teuthology.orchestra.run.smithi057.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-25T23:46:06.293 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-25T23:46:06.293 INFO:teuthology.orchestra.run.smithi057.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-25T23:46:06.293 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-04-25T23:46:06.293 INFO:teuthology.orchestra.run.smithi057.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-04-25T23:46:06.293 INFO:teuthology.orchestra.run.smithi057.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-25T23:46:06.293 INFO:teuthology.orchestra.run.smithi057.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-25T23:46:06.293 INFO:teuthology.orchestra.run.smithi057.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-25T23:46:06.294 INFO:teuthology.orchestra.run.smithi057.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-25T23:46:06.294 INFO:teuthology.orchestra.run.smithi057.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-04-25T23:46:06.294 INFO:teuthology.orchestra.run.smithi057.stdout: xmlstarlet zip 2024-04-25T23:46:06.294 INFO:teuthology.orchestra.run.smithi057.stdout:Suggested packages: 2024-04-25T23:46:06.294 INFO:teuthology.orchestra.run.smithi057.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-04-25T23:46:06.294 INFO:teuthology.orchestra.run.smithi057.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-04-25T23:46:06.294 INFO:teuthology.orchestra.run.smithi057.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-04-25T23:46:06.294 INFO:teuthology.orchestra.run.smithi057.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-04-25T23:46:06.294 INFO:teuthology.orchestra.run.smithi057.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-04-25T23:46:06.294 INFO:teuthology.orchestra.run.smithi057.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-04-25T23:46:06.294 INFO:teuthology.orchestra.run.smithi057.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-04-25T23:46:06.294 INFO:teuthology.orchestra.run.smithi057.stdout:Recommended packages: 2024-04-25T23:46:06.294 INFO:teuthology.orchestra.run.smithi057.stdout: btrfs-tools 2024-04-25T23:46:06.315 INFO:teuthology.orchestra.run.smithi080.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T23:46:06.315 INFO:teuthology.orchestra.run.smithi080.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T23:46:06.315 INFO:teuthology.orchestra.run.smithi080.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T23:46:06.315 INFO:teuthology.orchestra.run.smithi080.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T23:46:06.315 INFO:teuthology.orchestra.run.smithi080.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T23:46:06.315 INFO:teuthology.orchestra.run.smithi080.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T23:46:06.315 INFO:teuthology.orchestra.run.smithi080.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T23:46:06.317 INFO:teuthology.orchestra.run.smithi080.stdout:The following additional packages will be installed: 2024-04-25T23:46:06.318 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-25T23:46:06.318 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-25T23:46:06.318 INFO:teuthology.orchestra.run.smithi080.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T23:46:06.318 INFO:teuthology.orchestra.run.smithi080.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-25T23:46:06.318 INFO:teuthology.orchestra.run.smithi080.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-04-25T23:46:06.318 INFO:teuthology.orchestra.run.smithi080.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-04-25T23:46:06.318 INFO:teuthology.orchestra.run.smithi080.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-25T23:46:06.318 INFO:teuthology.orchestra.run.smithi080.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-25T23:46:06.318 INFO:teuthology.orchestra.run.smithi080.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-25T23:46:06.319 INFO:teuthology.orchestra.run.smithi080.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-04-25T23:46:06.319 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-25T23:46:06.319 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-25T23:46:06.319 INFO:teuthology.orchestra.run.smithi080.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-25T23:46:06.319 INFO:teuthology.orchestra.run.smithi080.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-25T23:46:06.319 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-25T23:46:06.319 INFO:teuthology.orchestra.run.smithi080.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-25T23:46:06.319 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-04-25T23:46:06.320 INFO:teuthology.orchestra.run.smithi080.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-04-25T23:46:06.320 INFO:teuthology.orchestra.run.smithi080.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-25T23:46:06.320 INFO:teuthology.orchestra.run.smithi080.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-25T23:46:06.320 INFO:teuthology.orchestra.run.smithi080.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-25T23:46:06.320 INFO:teuthology.orchestra.run.smithi080.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-25T23:46:06.320 INFO:teuthology.orchestra.run.smithi080.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-04-25T23:46:06.320 INFO:teuthology.orchestra.run.smithi080.stdout: xmlstarlet zip 2024-04-25T23:46:06.321 INFO:teuthology.orchestra.run.smithi080.stdout:Suggested packages: 2024-04-25T23:46:06.321 INFO:teuthology.orchestra.run.smithi080.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-04-25T23:46:06.321 INFO:teuthology.orchestra.run.smithi080.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-04-25T23:46:06.321 INFO:teuthology.orchestra.run.smithi080.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-04-25T23:46:06.321 INFO:teuthology.orchestra.run.smithi080.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-04-25T23:46:06.321 INFO:teuthology.orchestra.run.smithi080.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-04-25T23:46:06.321 INFO:teuthology.orchestra.run.smithi080.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-04-25T23:46:06.321 INFO:teuthology.orchestra.run.smithi080.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-04-25T23:46:06.321 INFO:teuthology.orchestra.run.smithi080.stdout:Recommended packages: 2024-04-25T23:46:06.321 INFO:teuthology.orchestra.run.smithi080.stdout: btrfs-tools 2024-04-25T23:46:06.400 INFO:teuthology.orchestra.run.smithi057.stdout:The following NEW packages will be installed: 2024-04-25T23:46:06.400 INFO:teuthology.orchestra.run.smithi057.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-25T23:46:06.400 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-25T23:46:06.400 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-04-25T23:46:06.400 INFO:teuthology.orchestra.run.smithi057.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-04-25T23:46:06.401 INFO:teuthology.orchestra.run.smithi057.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-04-25T23:46:06.401 INFO:teuthology.orchestra.run.smithi057.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-04-25T23:46:06.402 INFO:teuthology.orchestra.run.smithi057.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-04-25T23:46:06.402 INFO:teuthology.orchestra.run.smithi057.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-04-25T23:46:06.402 INFO:teuthology.orchestra.run.smithi057.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-04-25T23:46:06.402 INFO:teuthology.orchestra.run.smithi057.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-04-25T23:46:06.403 INFO:teuthology.orchestra.run.smithi057.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-04-25T23:46:06.403 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-04-25T23:46:06.403 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-25T23:46:06.403 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-25T23:46:06.403 INFO:teuthology.orchestra.run.smithi057.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-25T23:46:06.403 INFO:teuthology.orchestra.run.smithi057.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-25T23:46:06.403 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-25T23:46:06.403 INFO:teuthology.orchestra.run.smithi057.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-25T23:46:06.403 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-04-25T23:46:06.403 INFO:teuthology.orchestra.run.smithi057.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-04-25T23:46:06.403 INFO:teuthology.orchestra.run.smithi057.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-04-25T23:46:06.403 INFO:teuthology.orchestra.run.smithi057.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-04-25T23:46:06.404 INFO:teuthology.orchestra.run.smithi057.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-04-25T23:46:06.404 INFO:teuthology.orchestra.run.smithi057.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-04-25T23:46:06.404 INFO:teuthology.orchestra.run.smithi057.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-04-25T23:46:06.404 INFO:teuthology.orchestra.run.smithi057.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-04-25T23:46:06.419 INFO:teuthology.orchestra.run.smithi080.stdout:The following NEW packages will be installed: 2024-04-25T23:46:06.419 INFO:teuthology.orchestra.run.smithi080.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-25T23:46:06.420 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-25T23:46:06.420 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-04-25T23:46:06.420 INFO:teuthology.orchestra.run.smithi080.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-04-25T23:46:06.420 INFO:teuthology.orchestra.run.smithi080.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-04-25T23:46:06.420 INFO:teuthology.orchestra.run.smithi080.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-04-25T23:46:06.421 INFO:teuthology.orchestra.run.smithi080.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-04-25T23:46:06.421 INFO:teuthology.orchestra.run.smithi080.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-04-25T23:46:06.421 INFO:teuthology.orchestra.run.smithi080.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-04-25T23:46:06.421 INFO:teuthology.orchestra.run.smithi080.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-04-25T23:46:06.422 INFO:teuthology.orchestra.run.smithi080.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-04-25T23:46:06.422 INFO:teuthology.orchestra.run.smithi080.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-04-25T23:46:06.422 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-25T23:46:06.422 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-25T23:46:06.422 INFO:teuthology.orchestra.run.smithi080.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-25T23:46:06.422 INFO:teuthology.orchestra.run.smithi080.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-25T23:46:06.422 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-25T23:46:06.422 INFO:teuthology.orchestra.run.smithi080.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-25T23:46:06.422 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-04-25T23:46:06.422 INFO:teuthology.orchestra.run.smithi080.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-04-25T23:46:06.422 INFO:teuthology.orchestra.run.smithi080.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-04-25T23:46:06.422 INFO:teuthology.orchestra.run.smithi080.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-04-25T23:46:06.422 INFO:teuthology.orchestra.run.smithi080.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-04-25T23:46:06.423 INFO:teuthology.orchestra.run.smithi080.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-04-25T23:46:06.423 INFO:teuthology.orchestra.run.smithi080.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-04-25T23:46:06.423 INFO:teuthology.orchestra.run.smithi080.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-04-25T23:46:06.469 INFO:teuthology.orchestra.run.smithi057.stdout:0 upgraded, 118 newly installed, 0 to remove and 302 not upgraded. 2024-04-25T23:46:06.469 INFO:teuthology.orchestra.run.smithi057.stdout:Need to get 179 MB of archives. 2024-04-25T23:46:06.469 INFO:teuthology.orchestra.run.smithi057.stdout:After this operation, 822 MB of additional disk space will be used. 2024-04-25T23:46:06.469 INFO:teuthology.orchestra.run.smithi057.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-25T23:46:06.487 INFO:teuthology.orchestra.run.smithi080.stdout:0 upgraded, 118 newly installed, 0 to remove and 302 not upgraded. 2024-04-25T23:46:06.487 INFO:teuthology.orchestra.run.smithi080.stdout:Need to get 179 MB of archives. 2024-04-25T23:46:06.487 INFO:teuthology.orchestra.run.smithi080.stdout:After this operation, 822 MB of additional disk space will be used. 2024-04-25T23:46:06.488 INFO:teuthology.orchestra.run.smithi080.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-25T23:46:06.563 INFO:teuthology.orchestra.run.smithi080.stdout:Get:2 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-2445-gd2c45d97-1jammy [3,541 kB] 2024-04-25T23:46:06.574 INFO:teuthology.orchestra.run.smithi057.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-25T23:46:06.578 INFO:teuthology.orchestra.run.smithi057.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-25T23:46:06.598 INFO:teuthology.orchestra.run.smithi057.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-25T23:46:06.601 INFO:teuthology.orchestra.run.smithi080.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-25T23:46:06.604 INFO:teuthology.orchestra.run.smithi080.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-25T23:46:06.625 INFO:teuthology.orchestra.run.smithi080.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-25T23:46:06.665 INFO:teuthology.orchestra.run.smithi057.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-25T23:46:06.669 INFO:teuthology.orchestra.run.smithi057.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-25T23:46:06.682 INFO:teuthology.orchestra.run.smithi057.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-25T23:46:06.686 INFO:teuthology.orchestra.run.smithi057.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-25T23:46:06.687 INFO:teuthology.orchestra.run.smithi057.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-25T23:46:06.687 INFO:teuthology.orchestra.run.smithi057.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-25T23:46:06.697 INFO:teuthology.orchestra.run.smithi057.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-04-25T23:46:06.699 INFO:teuthology.orchestra.run.smithi057.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-04-25T23:46:06.701 INFO:teuthology.orchestra.run.smithi057.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-04-25T23:46:06.708 INFO:teuthology.orchestra.run.smithi057.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-04-25T23:46:06.714 INFO:teuthology.orchestra.run.smithi080.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-25T23:46:06.721 INFO:teuthology.orchestra.run.smithi080.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-25T23:46:06.734 INFO:teuthology.orchestra.run.smithi057.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-04-25T23:46:06.735 INFO:teuthology.orchestra.run.smithi080.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-25T23:46:06.742 INFO:teuthology.orchestra.run.smithi057.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-04-25T23:46:06.743 INFO:teuthology.orchestra.run.smithi080.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-25T23:46:06.743 INFO:teuthology.orchestra.run.smithi080.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-25T23:46:06.743 INFO:teuthology.orchestra.run.smithi080.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-25T23:46:06.743 INFO:teuthology.orchestra.run.smithi057.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-04-25T23:46:06.747 INFO:teuthology.orchestra.run.smithi057.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-25T23:46:06.748 INFO:teuthology.orchestra.run.smithi057.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-25T23:46:06.749 INFO:teuthology.orchestra.run.smithi057.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-25T23:46:06.751 INFO:teuthology.orchestra.run.smithi057.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-25T23:46:06.751 INFO:teuthology.orchestra.run.smithi057.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-25T23:46:06.752 INFO:teuthology.orchestra.run.smithi080.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-04-25T23:46:06.752 INFO:teuthology.orchestra.run.smithi057.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-25T23:46:06.754 INFO:teuthology.orchestra.run.smithi080.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-04-25T23:46:06.756 INFO:teuthology.orchestra.run.smithi080.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-04-25T23:46:06.758 INFO:teuthology.orchestra.run.smithi080.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-04-25T23:46:06.766 INFO:teuthology.orchestra.run.smithi057.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-25T23:46:06.766 INFO:teuthology.orchestra.run.smithi057.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-25T23:46:06.772 INFO:teuthology.orchestra.run.smithi080.stdout:Get:16 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-2445-gd2c45d97-1jammy [3,218 kB] 2024-04-25T23:46:06.776 INFO:teuthology.orchestra.run.smithi080.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-04-25T23:46:06.779 INFO:teuthology.orchestra.run.smithi080.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-04-25T23:46:06.781 INFO:teuthology.orchestra.run.smithi080.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-04-25T23:46:06.783 INFO:teuthology.orchestra.run.smithi080.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-25T23:46:06.784 INFO:teuthology.orchestra.run.smithi080.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-25T23:46:06.784 INFO:teuthology.orchestra.run.smithi080.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-25T23:46:06.786 INFO:teuthology.orchestra.run.smithi057.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-25T23:46:06.787 INFO:teuthology.orchestra.run.smithi057.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-25T23:46:06.792 INFO:teuthology.orchestra.run.smithi057.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-25T23:46:06.793 INFO:teuthology.orchestra.run.smithi057.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-25T23:46:06.794 INFO:teuthology.orchestra.run.smithi057.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-25T23:46:06.795 INFO:teuthology.orchestra.run.smithi057.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-25T23:46:06.796 INFO:teuthology.orchestra.run.smithi080.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-25T23:46:06.797 INFO:teuthology.orchestra.run.smithi057.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-25T23:46:06.797 INFO:teuthology.orchestra.run.smithi080.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-25T23:46:06.797 INFO:teuthology.orchestra.run.smithi080.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-25T23:46:06.798 INFO:teuthology.orchestra.run.smithi057.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-25T23:46:06.798 INFO:teuthology.orchestra.run.smithi080.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-25T23:46:06.807 INFO:teuthology.orchestra.run.smithi057.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-25T23:46:06.809 INFO:teuthology.orchestra.run.smithi057.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-25T23:46:06.817 INFO:teuthology.orchestra.run.smithi080.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-25T23:46:06.818 INFO:teuthology.orchestra.run.smithi080.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-25T23:46:06.818 INFO:teuthology.orchestra.run.smithi080.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-25T23:46:06.821 INFO:teuthology.orchestra.run.smithi080.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-25T23:46:06.821 INFO:teuthology.orchestra.run.smithi080.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-25T23:46:06.822 INFO:teuthology.orchestra.run.smithi080.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-25T23:46:06.828 INFO:teuthology.orchestra.run.smithi057.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-25T23:46:06.829 INFO:teuthology.orchestra.run.smithi057.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-25T23:46:06.838 INFO:teuthology.orchestra.run.smithi080.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-25T23:46:06.840 INFO:teuthology.orchestra.run.smithi057.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-25T23:46:06.840 INFO:teuthology.orchestra.run.smithi057.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-25T23:46:06.840 INFO:teuthology.orchestra.run.smithi080.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-25T23:46:06.840 INFO:teuthology.orchestra.run.smithi080.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-25T23:46:06.840 INFO:teuthology.orchestra.run.smithi080.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-25T23:46:06.841 INFO:teuthology.orchestra.run.smithi057.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-25T23:46:06.842 INFO:teuthology.orchestra.run.smithi057.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-25T23:46:06.844 INFO:teuthology.orchestra.run.smithi057.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-25T23:46:06.848 INFO:teuthology.orchestra.run.smithi057.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-25T23:46:06.850 INFO:teuthology.orchestra.run.smithi057.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-25T23:46:06.852 INFO:teuthology.orchestra.run.smithi057.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-25T23:46:06.858 INFO:teuthology.orchestra.run.smithi080.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-25T23:46:06.860 INFO:teuthology.orchestra.run.smithi080.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-25T23:46:06.860 INFO:teuthology.orchestra.run.smithi080.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-25T23:46:06.865 INFO:teuthology.orchestra.run.smithi080.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-25T23:46:06.866 INFO:teuthology.orchestra.run.smithi080.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-25T23:46:06.866 INFO:teuthology.orchestra.run.smithi080.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-25T23:46:06.869 INFO:teuthology.orchestra.run.smithi057.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-25T23:46:06.871 INFO:teuthology.orchestra.run.smithi080.stdout:Get:43 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-2445-gd2c45d97-1jammy [711 kB] 2024-04-25T23:46:06.873 INFO:teuthology.orchestra.run.smithi057.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-25T23:46:06.874 INFO:teuthology.orchestra.run.smithi057.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-25T23:46:06.876 INFO:teuthology.orchestra.run.smithi057.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-25T23:46:06.881 INFO:teuthology.orchestra.run.smithi057.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-25T23:46:06.893 INFO:teuthology.orchestra.run.smithi080.stdout:Get:44 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-2445-gd2c45d97-1jammy [356 kB] 2024-04-25T23:46:06.894 INFO:teuthology.orchestra.run.smithi080.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-25T23:46:06.896 INFO:teuthology.orchestra.run.smithi080.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-25T23:46:06.900 INFO:teuthology.orchestra.run.smithi080.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-25T23:46:06.902 INFO:teuthology.orchestra.run.smithi080.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-25T23:46:06.905 INFO:teuthology.orchestra.run.smithi080.stdout:Get:49 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-2445-gd2c45d97-1jammy [32.8 kB] 2024-04-25T23:46:06.906 INFO:teuthology.orchestra.run.smithi080.stdout:Get:50 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-2445-gd2c45d97-1jammy [180 kB] 2024-04-25T23:46:06.911 INFO:teuthology.orchestra.run.smithi080.stdout:Get:51 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-2445-gd2c45d97-1jammy [65.8 kB] 2024-04-25T23:46:06.914 INFO:teuthology.orchestra.run.smithi080.stdout:Get:52 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-2445-gd2c45d97-1jammy [330 kB] 2024-04-25T23:46:06.925 INFO:teuthology.orchestra.run.smithi080.stdout:Get:53 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-2445-gd2c45d97-1jammy [6,775 kB] 2024-04-25T23:46:06.929 INFO:teuthology.orchestra.run.smithi080.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-25T23:46:06.930 INFO:teuthology.orchestra.run.smithi080.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-25T23:46:06.935 INFO:teuthology.orchestra.run.smithi080.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-25T23:46:06.936 INFO:teuthology.orchestra.run.smithi080.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-25T23:46:06.938 INFO:teuthology.orchestra.run.smithi080.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-25T23:46:06.943 INFO:teuthology.orchestra.run.smithi080.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-25T23:46:06.971 INFO:teuthology.orchestra.run.smithi057.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-25T23:46:06.972 INFO:teuthology.orchestra.run.smithi057.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-25T23:46:06.975 INFO:teuthology.orchestra.run.smithi057.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-25T23:46:06.976 INFO:teuthology.orchestra.run.smithi057.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-25T23:46:06.978 INFO:teuthology.orchestra.run.smithi057.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-25T23:46:07.083 INFO:teuthology.orchestra.run.smithi057.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-25T23:46:07.088 INFO:teuthology.orchestra.run.smithi057.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-25T23:46:07.088 INFO:teuthology.orchestra.run.smithi057.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-25T23:46:07.100 INFO:teuthology.orchestra.run.smithi080.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-25T23:46:07.102 INFO:teuthology.orchestra.run.smithi080.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-25T23:46:07.104 INFO:teuthology.orchestra.run.smithi080.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-25T23:46:07.104 INFO:teuthology.orchestra.run.smithi080.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-25T23:46:07.106 INFO:teuthology.orchestra.run.smithi080.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-25T23:46:07.122 INFO:teuthology.orchestra.run.smithi057.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-25T23:46:07.123 INFO:teuthology.orchestra.run.smithi057.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-25T23:46:07.124 INFO:teuthology.orchestra.run.smithi057.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-25T23:46:07.125 INFO:teuthology.orchestra.run.smithi057.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-25T23:46:07.125 INFO:teuthology.orchestra.run.smithi057.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-25T23:46:07.126 INFO:teuthology.orchestra.run.smithi057.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-25T23:46:07.127 INFO:teuthology.orchestra.run.smithi057.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-25T23:46:07.135 INFO:teuthology.orchestra.run.smithi080.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-25T23:46:07.138 INFO:teuthology.orchestra.run.smithi080.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-25T23:46:07.138 INFO:teuthology.orchestra.run.smithi080.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-25T23:46:07.150 INFO:teuthology.orchestra.run.smithi080.stdout:Get:68 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-2445-gd2c45d97-1jammy [112 kB] 2024-04-25T23:46:07.150 INFO:teuthology.orchestra.run.smithi057.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-25T23:46:07.151 INFO:teuthology.orchestra.run.smithi080.stdout:Get:69 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-2445-gd2c45d97-1jammy [467 kB] 2024-04-25T23:46:07.154 INFO:teuthology.orchestra.run.smithi057.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-25T23:46:07.157 INFO:teuthology.orchestra.run.smithi080.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-25T23:46:07.157 INFO:teuthology.orchestra.run.smithi080.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-25T23:46:07.162 INFO:teuthology.orchestra.run.smithi080.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-25T23:46:07.163 INFO:teuthology.orchestra.run.smithi080.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-25T23:46:07.163 INFO:teuthology.orchestra.run.smithi080.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-25T23:46:07.164 INFO:teuthology.orchestra.run.smithi080.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-25T23:46:07.174 INFO:teuthology.orchestra.run.smithi057.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-25T23:46:07.176 INFO:teuthology.orchestra.run.smithi057.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-25T23:46:07.177 INFO:teuthology.orchestra.run.smithi057.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-25T23:46:07.183 INFO:teuthology.orchestra.run.smithi057.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-25T23:46:07.187 INFO:teuthology.orchestra.run.smithi057.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-04-25T23:46:07.188 INFO:teuthology.orchestra.run.smithi057.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-04-25T23:46:07.189 INFO:teuthology.orchestra.run.smithi057.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-25T23:46:07.193 INFO:teuthology.orchestra.run.smithi080.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-25T23:46:07.196 INFO:teuthology.orchestra.run.smithi057.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-04-25T23:46:07.205 INFO:teuthology.orchestra.run.smithi057.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-25T23:46:07.210 INFO:teuthology.orchestra.run.smithi057.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-25T23:46:07.213 INFO:teuthology.orchestra.run.smithi080.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-25T23:46:07.213 INFO:teuthology.orchestra.run.smithi080.stdout:Get:78 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-2445-gd2c45d97-1jammy [25.6 MB] 2024-04-25T23:46:07.218 INFO:teuthology.orchestra.run.smithi080.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-25T23:46:07.221 INFO:teuthology.orchestra.run.smithi080.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-25T23:46:07.223 INFO:teuthology.orchestra.run.smithi080.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-25T23:46:07.224 INFO:teuthology.orchestra.run.smithi080.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-25T23:46:07.227 INFO:teuthology.orchestra.run.smithi057.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-25T23:46:07.228 INFO:teuthology.orchestra.run.smithi057.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-25T23:46:07.228 INFO:teuthology.orchestra.run.smithi057.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-25T23:46:07.230 INFO:teuthology.orchestra.run.smithi080.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-25T23:46:07.231 INFO:teuthology.orchestra.run.smithi057.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-25T23:46:07.232 INFO:teuthology.orchestra.run.smithi057.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-25T23:46:07.233 INFO:teuthology.orchestra.run.smithi080.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-04-25T23:46:07.235 INFO:teuthology.orchestra.run.smithi080.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-04-25T23:46:07.236 INFO:teuthology.orchestra.run.smithi080.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-25T23:46:07.243 INFO:teuthology.orchestra.run.smithi057.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-25T23:46:07.244 INFO:teuthology.orchestra.run.smithi057.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-25T23:46:07.244 INFO:teuthology.orchestra.run.smithi057.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-25T23:46:07.245 INFO:teuthology.orchestra.run.smithi080.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-04-25T23:46:07.248 INFO:teuthology.orchestra.run.smithi057.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-25T23:46:07.249 INFO:teuthology.orchestra.run.smithi057.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-25T23:46:07.276 INFO:teuthology.orchestra.run.smithi080.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-25T23:46:07.282 INFO:teuthology.orchestra.run.smithi080.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-25T23:46:07.283 INFO:teuthology.orchestra.run.smithi080.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-25T23:46:07.284 INFO:teuthology.orchestra.run.smithi080.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-25T23:46:07.285 INFO:teuthology.orchestra.run.smithi080.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-25T23:46:07.288 INFO:teuthology.orchestra.run.smithi080.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-25T23:46:07.289 INFO:teuthology.orchestra.run.smithi080.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-25T23:46:07.302 INFO:teuthology.orchestra.run.smithi080.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-25T23:46:07.303 INFO:teuthology.orchestra.run.smithi080.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-25T23:46:07.303 INFO:teuthology.orchestra.run.smithi080.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-25T23:46:07.323 INFO:teuthology.orchestra.run.smithi080.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-25T23:46:07.325 INFO:teuthology.orchestra.run.smithi080.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-25T23:46:07.708 INFO:teuthology.orchestra.run.smithi057.stdout:Get:88 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-2445-gd2c45d97-1jammy [3,541 kB] 2024-04-25T23:46:08.142 INFO:teuthology.orchestra.run.smithi080.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-2445-gd2c45d97-1jammy [5,084 kB] 2024-04-25T23:46:08.220 INFO:teuthology.orchestra.run.smithi080.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-2445-gd2c45d97-1jammy [244 kB] 2024-04-25T23:46:08.223 INFO:teuthology.orchestra.run.smithi080.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-2445-gd2c45d97-1jammy [124 kB] 2024-04-25T23:46:08.224 INFO:teuthology.orchestra.run.smithi080.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-2445-gd2c45d97-1jammy [1,526 kB] 2024-04-25T23:46:08.269 INFO:teuthology.orchestra.run.smithi080.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-2445-gd2c45d97-1jammy [6,245 kB] 2024-04-25T23:46:08.489 INFO:teuthology.orchestra.run.smithi080.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-2445-gd2c45d97-1jammy [22.7 MB] 2024-04-25T23:46:09.374 INFO:teuthology.orchestra.run.smithi057.stdout:Get:89 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-2445-gd2c45d97-1jammy [3,218 kB] 2024-04-25T23:46:09.404 INFO:teuthology.orchestra.run.smithi080.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-2445-gd2c45d97-1jammy [14.0 kB] 2024-04-25T23:46:09.405 INFO:teuthology.orchestra.run.smithi080.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-2445-gd2c45d97-1jammy [869 kB] 2024-04-25T23:46:09.422 INFO:teuthology.orchestra.run.smithi080.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-2445-gd2c45d97-1jammy [2,437 kB] 2024-04-25T23:46:09.463 INFO:teuthology.orchestra.run.smithi080.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-2445-gd2c45d97-1jammy [791 kB] 2024-04-25T23:46:09.473 INFO:teuthology.orchestra.run.smithi080.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-2445-gd2c45d97-1jammy [147 kB] 2024-04-25T23:46:09.476 INFO:teuthology.orchestra.run.smithi080.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-2445-gd2c45d97-1jammy [2,293 kB] 2024-04-25T23:46:09.547 INFO:teuthology.orchestra.run.smithi080.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-2445-gd2c45d97-1jammy [8,625 kB] 2024-04-25T23:46:09.880 INFO:teuthology.orchestra.run.smithi080.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-2445-gd2c45d97-1jammy [14.2 kB] 2024-04-25T23:46:09.880 INFO:teuthology.orchestra.run.smithi080.stdout:Get:114 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-2445-gd2c45d97-1jammy [50.7 MB] 2024-04-25T23:46:10.109 INFO:teuthology.orchestra.run.smithi057.stdout:Get:90 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-2445-gd2c45d97-1jammy [711 kB] 2024-04-25T23:46:10.235 INFO:teuthology.orchestra.run.smithi057.stdout:Get:91 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-2445-gd2c45d97-1jammy [356 kB] 2024-04-25T23:46:10.347 INFO:teuthology.orchestra.run.smithi057.stdout:Get:92 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-2445-gd2c45d97-1jammy [32.8 kB] 2024-04-25T23:46:10.348 INFO:teuthology.orchestra.run.smithi057.stdout:Get:93 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-2445-gd2c45d97-1jammy [180 kB] 2024-04-25T23:46:10.405 INFO:teuthology.orchestra.run.smithi057.stdout:Get:94 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-2445-gd2c45d97-1jammy [65.8 kB] 2024-04-25T23:46:10.407 INFO:teuthology.orchestra.run.smithi057.stdout:Get:95 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-2445-gd2c45d97-1jammy [330 kB] 2024-04-25T23:46:10.470 INFO:teuthology.orchestra.run.smithi057.stdout:Get:96 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-2445-gd2c45d97-1jammy [6,775 kB] 2024-04-25T23:46:11.934 INFO:teuthology.orchestra.run.smithi057.stdout:Get:97 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-2445-gd2c45d97-1jammy [112 kB] 2024-04-25T23:46:11.969 INFO:teuthology.orchestra.run.smithi057.stdout:Get:98 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-2445-gd2c45d97-1jammy [467 kB] 2024-04-25T23:46:12.006 INFO:teuthology.orchestra.run.smithi057.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-2445-gd2c45d97-1jammy [25.6 MB] 2024-04-25T23:46:12.173 INFO:teuthology.orchestra.run.smithi080.stdout:Get:115 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-2445-gd2c45d97-1jammy [121 kB] 2024-04-25T23:46:12.176 INFO:teuthology.orchestra.run.smithi080.stdout:Get:116 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-2445-gd2c45d97-1jammy [38.9 kB] 2024-04-25T23:46:12.177 INFO:teuthology.orchestra.run.smithi080.stdout:Get:117 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-2445-gd2c45d97-1jammy [13.4 MB] 2024-04-25T23:46:12.564 INFO:teuthology.orchestra.run.smithi080.stdout:Get:118 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-2445-gd2c45d97-1jammy [91.4 kB] 2024-04-25T23:46:13.118 INFO:teuthology.orchestra.run.smithi080.stdout:Fetched 179 MB in 6s (29.3 MB/s) 2024-04-25T23:46:13.277 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-25T23:46:13.377 INFO:teuthology.orchestra.run.smithi080.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165995 files and directories currently installed.) 2024-04-25T23:46:13.383 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-25T23:46:13.468 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T23:46:13.813 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-25T23:46:13.839 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-25T23:46:13.879 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T23:46:13.993 INFO:teuthology.orchestra.run.smithi057.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-2445-gd2c45d97-1jammy [5,084 kB] 2024-04-25T23:46:14.199 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-25T23:46:14.225 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-25T23:46:14.266 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T23:46:14.288 INFO:teuthology.orchestra.run.smithi057.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-2445-gd2c45d97-1jammy [244 kB] 2024-04-25T23:46:14.293 INFO:teuthology.orchestra.run.smithi057.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-2445-gd2c45d97-1jammy [124 kB] 2024-04-25T23:46:14.309 INFO:teuthology.orchestra.run.smithi057.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-2445-gd2c45d97-1jammy [1,526 kB] 2024-04-25T23:46:14.396 INFO:teuthology.orchestra.run.smithi057.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-2445-gd2c45d97-1jammy [6,245 kB] 2024-04-25T23:46:14.686 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-25T23:46:14.712 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T23:46:14.761 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T23:46:14.772 INFO:teuthology.orchestra.run.smithi057.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-2445-gd2c45d97-1jammy [22.7 MB] 2024-04-25T23:46:15.131 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-25T23:46:15.157 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T23:46:15.197 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T23:46:15.534 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-25T23:46:15.561 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T23:46:15.601 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T23:46:15.938 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-25T23:46:15.964 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-25T23:46:16.004 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T23:46:16.167 INFO:teuthology.orchestra.run.smithi057.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-2445-gd2c45d97-1jammy [14.0 kB] 2024-04-25T23:46:16.168 INFO:teuthology.orchestra.run.smithi057.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-2445-gd2c45d97-1jammy [869 kB] 2024-04-25T23:46:16.184 INFO:teuthology.orchestra.run.smithi057.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-2445-gd2c45d97-1jammy [2,437 kB] 2024-04-25T23:46:16.281 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package librados2. 2024-04-25T23:46:16.297 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../007-librados2_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T23:46:16.315 INFO:teuthology.orchestra.run.smithi057.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-2445-gd2c45d97-1jammy [791 kB] 2024-04-25T23:46:16.331 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking librados2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:46:16.357 INFO:teuthology.orchestra.run.smithi057.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-2445-gd2c45d97-1jammy [147 kB] 2024-04-25T23:46:16.360 INFO:teuthology.orchestra.run.smithi057.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-2445-gd2c45d97-1jammy [2,293 kB] 2024-04-25T23:46:16.482 INFO:teuthology.orchestra.run.smithi057.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-2445-gd2c45d97-1jammy [8,625 kB] 2024-04-25T23:46:16.801 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package librbd1. 2024-04-25T23:46:16.828 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../008-librbd1_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T23:46:16.868 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking librbd1 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:46:16.919 INFO:teuthology.orchestra.run.smithi057.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-2445-gd2c45d97-1jammy [14.2 kB] 2024-04-25T23:46:16.920 INFO:teuthology.orchestra.run.smithi057.stdout:Get:114 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-2445-gd2c45d97-1jammy [50.7 MB] 2024-04-25T23:46:17.304 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libcephfs2. 2024-04-25T23:46:17.331 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../009-libcephfs2_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T23:46:17.371 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libcephfs2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:46:17.690 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-rados. 2024-04-25T23:46:17.716 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../010-python3-rados_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T23:46:17.956 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-rados (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:46:18.251 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-25T23:46:18.278 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T23:46:18.321 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-ceph-argparse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:46:18.570 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-cephfs. 2024-04-25T23:46:18.598 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../012-python3-cephfs_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T23:46:18.637 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-cephfs (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:46:18.954 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-25T23:46:18.975 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../013-python3-ceph-common_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T23:46:19.014 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-ceph-common (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:46:19.309 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-25T23:46:19.336 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-25T23:46:19.376 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T23:46:19.484 INFO:teuthology.orchestra.run.smithi057.stdout:Get:115 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-2445-gd2c45d97-1jammy [121 kB] 2024-04-25T23:46:19.487 INFO:teuthology.orchestra.run.smithi057.stdout:Get:116 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-2445-gd2c45d97-1jammy [38.9 kB] 2024-04-25T23:46:19.489 INFO:teuthology.orchestra.run.smithi057.stdout:Get:117 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-2445-gd2c45d97-1jammy [13.4 MB] 2024-04-25T23:46:19.662 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-prettytable. 2024-04-25T23:46:19.689 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-04-25T23:46:19.729 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-25T23:46:19.990 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-rbd. 2024-04-25T23:46:20.016 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../016-python3-rbd_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T23:46:20.056 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-rbd (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:46:20.375 INFO:teuthology.orchestra.run.smithi057.stdout:Get:118 https://4.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-2445-gd2c45d97-1jammy [91.4 kB] 2024-04-25T23:46:20.401 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-25T23:46:20.428 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-25T23:46:20.468 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T23:46:20.729 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-04-25T23:46:20.744 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-04-25T23:46:20.778 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T23:46:20.957 INFO:teuthology.orchestra.run.smithi057.stdout:Fetched 179 MB in 14s (12.9 MB/s) 2024-04-25T23:46:21.078 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-04-25T23:46:21.089 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-25T23:46:21.105 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-04-25T23:46:21.148 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T23:46:21.190 INFO:teuthology.orchestra.run.smithi057.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165995 files and directories currently installed.) 2024-04-25T23:46:21.196 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-25T23:46:21.271 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T23:46:21.485 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package lua5.1. 2024-04-25T23:46:21.512 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-04-25T23:46:21.552 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-04-25T23:46:21.608 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-25T23:46:21.634 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-25T23:46:21.674 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T23:46:21.863 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package lua-any. 2024-04-25T23:46:21.878 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-04-25T23:46:21.913 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking lua-any (27ubuntu1) ... 2024-04-25T23:46:22.002 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-25T23:46:22.029 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-25T23:46:22.069 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T23:46:22.133 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package zip. 2024-04-25T23:46:22.148 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-04-25T23:46:22.191 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking zip (3.0-12build2) ... 2024-04-25T23:46:22.464 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-25T23:46:22.490 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T23:46:22.511 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package unzip. 2024-04-25T23:46:22.530 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T23:46:22.537 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-04-25T23:46:22.586 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-04-25T23:46:22.942 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-25T23:46:22.968 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T23:46:22.981 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package luarocks. 2024-04-25T23:46:23.007 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-04-25T23:46:23.048 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-04-25T23:46:23.142 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T23:46:23.478 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-25T23:46:23.498 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package librgw2. 2024-04-25T23:46:23.504 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T23:46:23.524 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../025-librgw2_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T23:46:23.545 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T23:46:23.556 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking librgw2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:46:24.022 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-25T23:46:24.049 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-25T23:46:24.089 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T23:46:24.100 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-rgw. 2024-04-25T23:46:24.115 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../026-python3-rgw_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T23:46:24.159 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-rgw (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:46:24.409 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package librados2. 2024-04-25T23:46:24.436 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../007-librados2_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T23:46:24.475 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking librados2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:46:24.495 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-25T23:46:24.521 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-25T23:46:24.561 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T23:46:24.898 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libradosstriper1. 2024-04-25T23:46:24.911 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package librbd1. 2024-04-25T23:46:24.924 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../028-libradosstriper1_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T23:46:24.927 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../008-librbd1_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T23:46:24.961 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking librbd1 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:46:24.964 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libradosstriper1 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:46:25.360 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package ceph-common. 2024-04-25T23:46:25.386 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../029-ceph-common_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T23:46:25.406 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package libcephfs2. 2024-04-25T23:46:25.432 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../009-libcephfs2_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T23:46:25.434 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking ceph-common (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:46:25.472 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking libcephfs2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:46:25.758 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-rados. 2024-04-25T23:46:25.785 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../010-python3-rados_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T23:46:25.824 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-rados (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:46:26.161 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-25T23:46:26.188 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T23:46:26.227 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-ceph-argparse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:46:26.446 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-cephfs. 2024-04-25T23:46:26.463 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../012-python3-cephfs_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T23:46:26.496 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-cephfs (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:46:26.756 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package ceph-base. 2024-04-25T23:46:26.783 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../030-ceph-base_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T23:46:26.799 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-25T23:46:26.826 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../013-python3-ceph-common_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T23:46:26.833 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking ceph-base (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:46:26.866 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-ceph-common (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:46:27.210 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-25T23:46:27.237 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-25T23:46:27.277 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T23:46:27.411 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-25T23:46:27.438 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-25T23:46:27.477 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-25T23:46:27.596 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-prettytable. 2024-04-25T23:46:27.623 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-04-25T23:46:27.663 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-25T23:46:27.755 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-cheroot. 2024-04-25T23:46:27.770 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-25T23:46:27.805 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T23:46:27.898 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-rbd. 2024-04-25T23:46:27.914 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../016-python3-rbd_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T23:46:27.948 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-rbd (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:46:28.109 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-25T23:46:28.135 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-25T23:46:28.175 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-25T23:46:28.276 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-25T23:46:28.303 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-25T23:46:28.343 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T23:46:28.620 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-04-25T23:46:28.636 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-04-25T23:46:28.678 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T23:46:28.686 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-25T23:46:28.713 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-25T23:46:28.753 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-25T23:46:29.056 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-25T23:46:29.082 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-25T23:46:29.122 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-25T23:46:29.148 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-04-25T23:46:29.164 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-04-25T23:46:29.206 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T23:46:29.400 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-tempora. 2024-04-25T23:46:29.427 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-04-25T23:46:29.467 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-25T23:46:29.518 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package lua5.1. 2024-04-25T23:46:29.544 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-04-25T23:46:29.576 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-04-25T23:46:29.737 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-portend. 2024-04-25T23:46:29.763 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-04-25T23:46:29.803 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-25T23:46:29.896 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package lua-any. 2024-04-25T23:46:29.912 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-04-25T23:46:29.954 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking lua-any (27ubuntu1) ... 2024-04-25T23:46:30.106 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-25T23:46:30.132 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-25T23:46:30.173 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-25T23:46:30.206 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package zip. 2024-04-25T23:46:30.222 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-04-25T23:46:30.264 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking zip (3.0-12build2) ... 2024-04-25T23:46:30.476 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-25T23:46:30.502 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-25T23:46:30.542 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-25T23:46:30.584 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package unzip. 2024-04-25T23:46:30.611 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-04-25T23:46:30.651 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-04-25T23:46:30.855 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-natsort. 2024-04-25T23:46:30.881 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-04-25T23:46:30.913 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-25T23:46:31.013 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package luarocks. 2024-04-25T23:46:31.040 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-04-25T23:46:31.079 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-04-25T23:46:31.224 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-logutils. 2024-04-25T23:46:31.251 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-04-25T23:46:31.291 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-25T23:46:31.495 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package librgw2. 2024-04-25T23:46:31.522 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../025-librgw2_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T23:46:31.562 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking librgw2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:46:31.569 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-25T23:46:31.584 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-25T23:46:31.627 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-25T23:46:31.955 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-mako. 2024-04-25T23:46:31.981 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-25T23:46:32.021 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T23:46:32.114 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-rgw. 2024-04-25T23:46:32.130 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../026-python3-rgw_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T23:46:32.172 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-rgw (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:46:32.342 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-25T23:46:32.368 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-25T23:46:32.408 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-25T23:46:32.525 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-25T23:46:32.552 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-25T23:46:32.592 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T23:46:32.714 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-25T23:46:32.738 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-25T23:46:32.769 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-25T23:46:32.895 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package libradosstriper1. 2024-04-25T23:46:32.921 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../028-libradosstriper1_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T23:46:32.961 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking libradosstriper1 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:46:33.056 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-webob. 2024-04-25T23:46:33.082 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-25T23:46:33.122 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-25T23:46:33.389 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package ceph-common. 2024-04-25T23:46:33.416 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../029-ceph-common_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T23:46:33.456 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking ceph-common (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:46:33.475 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-waitress. 2024-04-25T23:46:33.502 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-25T23:46:33.545 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T23:46:33.879 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-tempita. 2024-04-25T23:46:33.905 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-25T23:46:33.946 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T23:46:34.273 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-paste. 2024-04-25T23:46:34.299 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-25T23:46:34.339 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T23:46:34.651 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-25T23:46:34.667 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-25T23:46:34.709 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T23:46:34.936 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package ceph-base. 2024-04-25T23:46:34.963 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../030-ceph-base_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T23:46:34.996 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-25T23:46:35.012 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking ceph-base (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:46:35.022 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-25T23:46:35.062 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-25T23:46:35.357 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-webtest. 2024-04-25T23:46:35.383 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-04-25T23:46:35.440 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-25T23:46:35.574 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-25T23:46:35.601 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-25T23:46:35.632 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-25T23:46:35.710 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-pecan. 2024-04-25T23:46:35.725 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-25T23:46:35.760 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T23:46:35.901 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-cheroot. 2024-04-25T23:46:35.928 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-25T23:46:35.967 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T23:46:36.073 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-certifi. 2024-04-25T23:46:36.100 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-04-25T23:46:36.140 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-25T23:46:36.295 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-25T23:46:36.322 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-25T23:46:36.362 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-25T23:46:36.460 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-idna. 2024-04-25T23:46:36.486 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../055-python3-idna_3.3-1_all.deb ... 2024-04-25T23:46:36.526 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-25T23:46:36.665 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-25T23:46:36.691 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-25T23:46:36.731 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-25T23:46:36.837 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-urllib3. 2024-04-25T23:46:36.864 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-25T23:46:36.904 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T23:46:37.018 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-25T23:46:37.044 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-25T23:46:37.084 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-25T23:46:37.232 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-requests. 2024-04-25T23:46:37.259 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-25T23:46:37.299 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T23:46:37.362 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-tempora. 2024-04-25T23:46:37.388 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-04-25T23:46:37.428 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-25T23:46:37.594 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-25T23:46:37.621 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-25T23:46:37.660 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T23:46:37.731 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-portend. 2024-04-25T23:46:37.758 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-04-25T23:46:37.797 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-25T23:46:38.100 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-25T23:46:38.127 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-25T23:46:38.167 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-25T23:46:38.420 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-25T23:46:38.435 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-25T23:46:38.469 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-25T23:46:38.797 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-25T23:46:38.812 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T23:46:38.815 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-natsort. 2024-04-25T23:46:38.842 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-04-25T23:46:38.855 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking ceph-mgr-modules-core (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:46:38.881 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-25T23:46:39.359 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-logutils. 2024-04-25T23:46:39.386 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-04-25T23:46:39.402 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-25T23:46:39.429 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T23:46:39.442 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-25T23:46:39.470 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:46:39.779 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-25T23:46:39.806 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-25T23:46:39.846 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-25T23:46:40.166 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-openssl. 2024-04-25T23:46:40.193 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-04-25T23:46:40.232 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-25T23:46:40.404 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-25T23:46:40.431 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-25T23:46:40.462 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-25T23:46:40.544 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package ceph-mgr. 2024-04-25T23:46:40.570 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../063-ceph-mgr_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T23:46:40.602 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking ceph-mgr (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:46:40.765 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-mako. 2024-04-25T23:46:40.792 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-25T23:46:40.831 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T23:46:40.997 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package ceph-mon. 2024-04-25T23:46:41.024 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../064-ceph-mon_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T23:46:41.063 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking ceph-mon (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:46:41.168 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-25T23:46:41.195 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-25T23:46:41.234 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-25T23:46:41.545 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-25T23:46:41.572 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-25T23:46:41.612 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-25T23:46:41.702 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package ceph-osd. 2024-04-25T23:46:41.728 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../065-ceph-osd_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T23:46:41.767 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking ceph-osd (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:46:41.873 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-webob. 2024-04-25T23:46:41.900 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-25T23:46:41.931 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-25T23:46:42.242 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-waitress. 2024-04-25T23:46:42.258 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-25T23:46:42.295 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T23:46:42.579 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-tempita. 2024-04-25T23:46:42.605 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-25T23:46:42.645 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T23:46:42.812 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package ceph. 2024-04-25T23:46:42.827 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../066-ceph_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T23:46:42.861 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking ceph (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:46:42.948 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-paste. 2024-04-25T23:46:42.975 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-25T23:46:43.014 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T23:46:43.198 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package ceph-fuse. 2024-04-25T23:46:43.225 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../067-ceph-fuse_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T23:46:43.264 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking ceph-fuse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:46:43.309 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-25T23:46:43.325 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-25T23:46:43.359 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T23:46:43.626 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package ceph-mds. 2024-04-25T23:46:43.653 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../068-ceph-mds_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T23:46:43.678 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-25T23:46:43.692 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking ceph-mds (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:46:43.705 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-25T23:46:43.745 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-25T23:46:44.048 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-webtest. 2024-04-25T23:46:44.075 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-04-25T23:46:44.079 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package cephadm. 2024-04-25T23:46:44.106 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../069-cephadm_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T23:46:44.114 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-25T23:46:44.137 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking cephadm (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:46:44.423 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-25T23:46:44.425 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-pecan. 2024-04-25T23:46:44.450 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-25T23:46:44.453 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-25T23:46:44.489 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-25T23:46:44.492 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T23:46:44.793 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-25T23:46:44.810 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T23:46:44.843 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T23:46:44.864 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-certifi. 2024-04-25T23:46:44.891 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-04-25T23:46:44.930 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-25T23:46:45.233 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-idna. 2024-04-25T23:46:45.260 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../055-python3-idna_3.3-1_all.deb ... 2024-04-25T23:46:45.291 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-25T23:46:45.561 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-urllib3. 2024-04-25T23:46:45.587 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-25T23:46:45.627 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T23:46:45.773 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-babel. 2024-04-25T23:46:45.800 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T23:46:45.839 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T23:46:45.930 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-requests. 2024-04-25T23:46:45.958 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-25T23:46:45.996 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T23:46:46.165 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-jinja2. 2024-04-25T23:46:46.193 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-25T23:46:46.225 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T23:46:46.308 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-25T23:46:46.335 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-25T23:46:46.374 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T23:46:46.537 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-25T23:46:46.564 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T23:46:46.595 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking ceph-mgr-cephadm (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:46:46.660 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-25T23:46:46.677 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T23:46:46.710 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking ceph-mgr-modules-core (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:46:46.915 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-25T23:46:46.941 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-04-25T23:46:46.981 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-25T23:46:47.099 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-25T23:46:47.125 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T23:46:47.165 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:46:47.243 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-routes. 2024-04-25T23:46:47.270 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-25T23:46:47.309 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T23:46:47.476 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-25T23:46:47.503 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-25T23:46:47.542 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-25T23:46:47.588 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-25T23:46:47.614 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T23:46:47.646 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking ceph-mgr-dashboard (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:46:47.829 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-openssl. 2024-04-25T23:46:47.845 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-04-25T23:46:47.878 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-25T23:46:48.215 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package ceph-mgr. 2024-04-25T23:46:48.242 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../063-ceph-mgr_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T23:46:48.281 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking ceph-mgr (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:46:48.676 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package ceph-mon. 2024-04-25T23:46:48.703 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../064-ceph-mon_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T23:46:48.742 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking ceph-mon (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:46:48.826 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-25T23:46:48.842 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-25T23:46:48.876 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T23:46:49.288 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package ceph-osd. 2024-04-25T23:46:49.315 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../065-ceph-osd_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T23:46:49.354 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking ceph-osd (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:46:49.496 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-joblib. 2024-04-25T23:46:49.524 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-25T23:46:49.562 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T23:46:49.958 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-25T23:46:49.985 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-25T23:46:50.024 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-25T23:46:50.327 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-sklearn. 2024-04-25T23:46:50.354 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-25T23:46:50.394 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T23:46:50.490 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package ceph. 2024-04-25T23:46:50.506 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../066-ceph_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T23:46:50.549 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking ceph (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:46:50.876 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package ceph-fuse. 2024-04-25T23:46:50.903 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../067-ceph-fuse_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T23:46:50.943 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking ceph-fuse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:46:51.138 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-25T23:46:51.165 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T23:46:51.271 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package ceph-mds. 2024-04-25T23:46:51.290 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:46:51.298 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../068-ceph-mds_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T23:46:51.329 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking ceph-mds (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:46:51.724 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package cephadm. 2024-04-25T23:46:51.751 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../069-cephadm_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T23:46:51.790 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking cephadm (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:46:52.134 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-25T23:46:52.158 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-25T23:46:52.201 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-25T23:46:52.227 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-cachetools. 2024-04-25T23:46:52.243 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-04-25T23:46:52.277 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-25T23:46:52.520 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-25T23:46:52.536 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T23:46:52.570 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T23:46:52.571 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-25T23:46:52.599 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-25T23:46:52.638 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-25T23:46:52.933 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-25T23:46:52.961 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-25T23:46:52.999 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T23:46:53.333 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-rsa. 2024-04-25T23:46:53.360 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-04-25T23:46:53.402 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-25T23:46:53.600 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-babel. 2024-04-25T23:46:53.628 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T23:46:53.666 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T23:46:53.723 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-google-auth. 2024-04-25T23:46:53.750 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-04-25T23:46:53.789 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-25T23:46:53.994 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-jinja2. 2024-04-25T23:46:54.022 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-25T23:46:54.069 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T23:46:54.110 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-25T23:46:54.138 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-25T23:46:54.176 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T23:46:54.463 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-websocket. 2024-04-25T23:46:54.472 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-25T23:46:54.490 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-04-25T23:46:54.500 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T23:46:54.521 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-25T23:46:54.538 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking ceph-mgr-cephadm (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:46:54.808 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-25T23:46:54.835 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-25T23:46:54.874 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T23:46:54.892 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-25T23:46:54.920 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-04-25T23:46:54.958 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-25T23:46:55.236 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-routes. 2024-04-25T23:46:55.263 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-25T23:46:55.305 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T23:46:55.547 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-25T23:46:55.575 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T23:46:55.599 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-25T23:46:55.613 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:46:55.627 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T23:46:55.657 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking ceph-mgr-dashboard (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:46:55.866 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-25T23:46:55.894 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-25T23:46:55.924 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T23:46:56.261 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-25T23:46:56.288 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T23:46:56.327 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T23:46:56.588 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package jq. 2024-04-25T23:46:56.604 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T23:46:56.646 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-25T23:46:56.874 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package socat. 2024-04-25T23:46:56.902 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-25T23:46:56.921 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-25T23:46:56.948 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-25T23:46:56.949 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-25T23:46:56.987 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T23:46:57.186 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package xmlstarlet. 2024-04-25T23:46:57.202 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-25T23:46:57.236 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-25T23:46:57.466 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-joblib. 2024-04-25T23:46:57.493 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-25T23:46:57.524 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T23:46:57.556 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package ceph-test. 2024-04-25T23:46:57.572 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../097-ceph-test_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T23:46:57.615 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking ceph-test (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:46:57.894 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-25T23:46:57.921 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-25T23:46:57.960 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-25T23:46:58.271 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-sklearn. 2024-04-25T23:46:58.299 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-25T23:46:58.346 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T23:46:59.071 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-25T23:46:59.099 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T23:46:59.281 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:00.119 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package ceph-volume. 2024-04-25T23:47:00.147 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../098-ceph-volume_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T23:47:00.186 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking ceph-volume (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:00.435 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-cachetools. 2024-04-25T23:47:00.463 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-04-25T23:47:00.489 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-25T23:47:00.493 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-25T23:47:00.516 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../099-libcephfs-dev_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T23:47:00.556 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libcephfs-dev (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:00.771 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-25T23:47:00.798 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-25T23:47:00.837 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-25T23:47:00.884 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package lua-socket:amd64. 2024-04-25T23:47:00.912 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-04-25T23:47:00.951 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T23:47:01.182 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-25T23:47:01.210 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-25T23:47:01.249 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T23:47:01.280 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package lua-sec:amd64. 2024-04-25T23:47:01.308 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-04-25T23:47:01.347 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-04-25T23:47:01.562 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-rsa. 2024-04-25T23:47:01.588 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-04-25T23:47:01.626 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-25T23:47:01.676 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package nvme-cli. 2024-04-25T23:47:01.703 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-25T23:47:01.734 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T23:47:01.939 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-google-auth. 2024-04-25T23:47:01.966 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-04-25T23:47:02.005 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-25T23:47:02.106 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package pkg-config. 2024-04-25T23:47:02.134 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-04-25T23:47:02.172 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T23:47:02.325 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-25T23:47:02.342 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-25T23:47:02.383 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T23:47:02.434 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-25T23:47:02.461 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-25T23:47:02.500 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-25T23:47:02.694 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-websocket. 2024-04-25T23:47:02.721 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-04-25T23:47:02.760 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-25T23:47:02.920 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-25T23:47:02.948 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-25T23:47:02.987 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-25T23:47:03.056 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-25T23:47:03.083 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-25T23:47:03.114 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T23:47:03.290 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-pastescript. 2024-04-25T23:47:03.317 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-04-25T23:47:03.356 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-25T23:47:03.676 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-pluggy. 2024-04-25T23:47:03.704 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-25T23:47:03.742 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-25T23:47:03.827 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-25T23:47:03.843 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T23:47:03.885 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:04.029 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-psutil. 2024-04-25T23:47:04.057 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-25T23:47:04.112 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-25T23:47:04.171 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-25T23:47:04.199 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-25T23:47:04.263 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T23:47:04.457 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-py. 2024-04-25T23:47:04.484 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-04-25T23:47:04.532 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-25T23:47:04.566 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-25T23:47:04.593 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T23:47:04.632 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T23:47:04.852 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-pygments. 2024-04-25T23:47:04.880 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-25T23:47:04.893 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package jq. 2024-04-25T23:47:04.909 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T23:47:04.918 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T23:47:04.951 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-25T23:47:05.204 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package socat. 2024-04-25T23:47:05.220 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-25T23:47:05.262 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-25T23:47:05.398 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-25T23:47:05.426 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-25T23:47:05.464 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-25T23:47:05.573 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package xmlstarlet. 2024-04-25T23:47:05.590 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-25T23:47:05.623 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-25T23:47:05.767 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-toml. 2024-04-25T23:47:05.795 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-04-25T23:47:05.833 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-25T23:47:05.994 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package ceph-test. 2024-04-25T23:47:06.022 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../097-ceph-test_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T23:47:06.060 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking ceph-test (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:06.120 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-pytest. 2024-04-25T23:47:06.147 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-25T23:47:06.178 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T23:47:06.532 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-simplejson. 2024-04-25T23:47:06.559 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-25T23:47:06.598 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-25T23:47:06.868 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-25T23:47:06.895 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-25T23:47:06.934 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-25T23:47:07.680 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package radosgw. 2024-04-25T23:47:07.708 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../116-radosgw_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T23:47:07.746 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking radosgw (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:08.390 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package ceph-volume. 2024-04-25T23:47:08.418 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../098-ceph-volume_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T23:47:08.448 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking ceph-volume (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:08.558 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package rbd-fuse. 2024-04-25T23:47:08.574 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../117-rbd-fuse_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T23:47:08.616 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking rbd-fuse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:08.735 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-25T23:47:08.751 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../099-libcephfs-dev_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T23:47:08.784 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking libcephfs-dev (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:08.904 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-25T23:47:09.037 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package lua-socket:amd64. 2024-04-25T23:47:09.065 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-04-25T23:47:09.095 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T23:47:09.154 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T23:47:09.281 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T23:47:09.450 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package lua-sec:amd64. 2024-04-25T23:47:09.466 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-04-25T23:47:09.508 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-04-25T23:47:09.541 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-25T23:47:09.859 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-25T23:47:09.887 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package nvme-cli. 2024-04-25T23:47:09.915 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-25T23:47:09.953 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T23:47:10.302 INFO:teuthology.orchestra.run.smithi080.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-25T23:47:10.366 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up cephadm (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:10.367 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package pkg-config. 2024-04-25T23:47:10.395 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-04-25T23:47:10.433 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T23:47:10.661 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-25T23:47:10.677 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-25T23:47:10.710 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-25T23:47:10.824 INFO:teuthology.orchestra.run.smithi080.stdout:Adding system user cephadm....done 2024-04-25T23:47:10.980 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T23:47:11.114 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-25T23:47:11.143 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-25T23:47:11.180 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-25T23:47:11.299 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-25T23:47:11.483 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-pastescript. 2024-04-25T23:47:11.511 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-04-25T23:47:11.549 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-25T23:47:11.558 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-25T23:47:11.684 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-25T23:47:11.878 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-pluggy. 2024-04-25T23:47:11.894 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-25T23:47:11.935 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-25T23:47:11.936 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-25T23:47:12.170 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T23:47:12.205 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-psutil. 2024-04-25T23:47:12.222 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-25T23:47:12.263 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-25T23:47:12.271 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-25T23:47:12.581 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T23:47:12.617 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-py. 2024-04-25T23:47:12.645 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-04-25T23:47:12.674 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-25T23:47:12.941 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-25T23:47:13.020 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-pygments. 2024-04-25T23:47:13.048 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-25T23:47:13.086 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T23:47:13.201 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-25T23:47:13.469 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-04-25T23:47:13.573 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-25T23:47:13.602 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-25T23:47:13.640 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-25T23:47:13.720 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-25T23:47:13.965 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-toml. 2024-04-25T23:47:13.980 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-25T23:47:13.993 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-04-25T23:47:14.034 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-25T23:47:14.248 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-25T23:47:14.311 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-pytest. 2024-04-25T23:47:14.328 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-25T23:47:14.361 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T23:47:14.490 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-ceph-argparse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:14.715 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-simplejson. 2024-04-25T23:47:14.744 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-25T23:47:14.781 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-25T23:47:14.833 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T23:47:14.959 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T23:47:15.042 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-25T23:47:15.058 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-25T23:47:15.085 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T23:47:15.100 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-25T23:47:15.211 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-25T23:47:15.612 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-04-25T23:47:16.018 INFO:teuthology.orchestra.run.smithi080.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-04-25T23:47:16.052 INFO:teuthology.orchestra.run.smithi080.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-04-25T23:47:16.071 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package radosgw. 2024-04-25T23:47:16.099 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../116-radosgw_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T23:47:16.124 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T23:47:16.128 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking radosgw (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:16.259 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-25T23:47:16.627 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T23:47:16.727 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-25T23:47:16.974 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package rbd-fuse. 2024-04-25T23:47:16.978 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T23:47:16.990 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../117-rbd-fuse_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T23:47:17.023 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking rbd-fuse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:17.246 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-25T23:47:17.320 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-25T23:47:17.564 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up zip (3.0-12build2) ... 2024-04-25T23:47:17.570 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T23:47:17.674 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T23:47:17.696 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T23:47:17.970 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-25T23:47:18.306 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-25T23:47:18.359 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T23:47:18.643 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T23:47:18.743 INFO:teuthology.orchestra.run.smithi057.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-25T23:47:18.744 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-25T23:47:18.814 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up cephadm (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:18.846 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T23:47:19.138 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-25T23:47:19.438 INFO:teuthology.orchestra.run.smithi057.stdout:Adding system user cephadm....done 2024-04-25T23:47:19.523 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-25T23:47:19.570 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T23:47:19.765 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T23:47:19.829 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-25T23:47:20.089 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-25T23:47:20.159 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T23:47:20.215 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-25T23:47:20.452 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T23:47:20.541 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-25T23:47:20.787 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-25T23:47:20.826 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T23:47:20.952 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-25T23:47:21.046 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-25T23:47:21.164 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-ceph-common (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:21.262 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T23:47:21.449 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-idna (3.3-1) ... 2024-04-25T23:47:21.622 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-25T23:47:21.750 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T23:47:21.873 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-25T23:47:22.133 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-04-25T23:47:22.392 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-25T23:47:22.652 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-25T23:47:22.903 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-25T23:47:22.922 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T23:47:23.137 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-ceph-argparse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:23.149 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T23:47:23.292 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-25T23:47:23.463 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T23:47:23.551 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T23:47:23.572 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T23:47:23.677 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-25T23:47:23.682 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T23:47:23.791 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-25T23:47:23.803 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T23:47:24.042 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-04-25T23:47:24.096 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-25T23:47:24.347 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-25T23:47:24.430 INFO:teuthology.orchestra.run.smithi057.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-04-25T23:47:24.458 INFO:teuthology.orchestra.run.smithi057.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-04-25T23:47:24.520 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T23:47:24.582 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T23:47:24.646 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-25T23:47:24.708 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-25T23:47:25.035 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-25T23:47:25.047 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T23:47:25.173 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-25T23:47:25.277 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-25T23:47:25.474 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T23:47:25.537 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-25T23:47:25.742 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-25T23:47:25.829 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-25T23:47:26.052 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up zip (3.0-12build2) ... 2024-04-25T23:47:26.072 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-25T23:47:26.161 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T23:47:26.332 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T23:47:26.458 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-25T23:47:26.742 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T23:47:26.843 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T23:47:26.888 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T23:47:27.111 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-25T23:47:27.164 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T23:47:27.265 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-25T23:47:27.374 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T23:47:27.404 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-25T23:47:27.664 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T23:47:27.675 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-25T23:47:27.773 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-04-25T23:47:27.882 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T23:47:27.972 INFO:teuthology.orchestra.run.smithi080.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-25T23:47:28.068 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-25T23:47:28.209 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T23:47:28.318 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T23:47:28.319 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T23:47:28.703 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-25T23:47:28.704 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T23:47:28.963 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up lua-any (27ubuntu1) ... 2024-04-25T23:47:28.988 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T23:47:29.072 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-25T23:47:29.315 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-25T23:47:29.348 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T23:47:29.474 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T23:47:29.566 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-25T23:47:29.692 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-ceph-common (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:29.793 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T23:47:30.010 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-idna (3.3-1) ... 2024-04-25T23:47:30.085 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T23:47:30.344 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T23:47:30.353 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-25T23:47:30.479 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-25T23:47:30.747 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-25T23:47:30.998 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T23:47:31.335 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-25T23:47:31.599 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T23:47:31.703 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-25T23:47:31.842 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T23:47:31.977 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-25T23:47:31.979 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T23:47:32.253 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T23:47:32.316 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T23:47:32.379 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-25T23:47:32.442 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up librados2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:32.505 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T23:47:32.568 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libsqlite3-mod-ceph (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:32.694 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-04-25T23:47:32.798 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-25T23:47:33.040 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-25T23:47:33.055 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libcephfs2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:33.164 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libradosstriper1 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:33.282 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-25T23:47:33.291 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T23:47:33.417 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-25T23:47:33.550 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up librbd1 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:33.677 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up ceph-mgr-modules-core (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:33.719 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-25T23:47:33.803 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up ceph-fuse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:33.978 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-25T23:47:34.017 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T23:47:34.017 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T23:47:34.246 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-25T23:47:34.496 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libcephfs-dev (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:34.505 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-25T23:47:34.622 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-rados (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:34.740 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-25T23:47:34.747 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up librgw2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:34.882 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T23:47:34.982 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T23:47:35.108 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-25T23:47:35.393 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T23:47:35.519 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T23:47:35.828 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-25T23:47:36.036 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-rbd (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:36.146 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-25T23:47:36.162 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up rbd-fuse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:36.280 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-rgw (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:36.405 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T23:47:36.422 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-cephfs (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:36.532 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up ceph-common (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:36.540 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-04-25T23:47:36.649 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T23:47:36.742 INFO:teuthology.orchestra.run.smithi057.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-25T23:47:37.034 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T23:47:37.075 INFO:teuthology.orchestra.run.smithi080.stdout:Adding group ceph....done 2024-04-25T23:47:37.160 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T23:47:37.451 INFO:teuthology.orchestra.run.smithi080.stdout:Adding system user ceph....done 2024-04-25T23:47:37.562 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-25T23:47:37.605 INFO:teuthology.orchestra.run.smithi080.stdout:Setting system user ceph properties....done 2024-04-25T23:47:37.618 INFO:teuthology.orchestra.run.smithi080.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-25T23:47:37.752 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-25T23:47:37.829 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up lua-any (27ubuntu1) ... 2024-04-25T23:47:37.955 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-25T23:47:38.060 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-25T23:47:38.198 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T23:47:38.307 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T23:47:38.485 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up ceph-test (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:38.611 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up radosgw (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:38.641 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T23:47:38.951 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T23:47:39.070 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T23:47:39.070 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T23:47:39.227 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-25T23:47:39.353 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-25T23:47:39.629 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-25T23:47:39.781 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up ceph-base (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:39.946 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T23:47:40.154 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-25T23:47:40.283 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-25T23:47:40.633 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up ceph-mds (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:40.668 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-25T23:47:40.859 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T23:47:40.859 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T23:47:40.961 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T23:47:41.307 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T23:47:41.433 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up librados2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:41.502 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up ceph-mgr (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:41.560 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up libsqlite3-mod-ceph (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:41.702 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-04-25T23:47:41.717 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T23:47:41.718 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T23:47:42.039 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up libcephfs2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:42.148 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up libradosstriper1 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:42.161 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up ceph-osd (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:42.274 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-25T23:47:42.435 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T23:47:42.436 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T23:47:42.551 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up librbd1 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:42.694 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up ceph-mgr-modules-core (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:42.820 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up ceph-fuse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:42.913 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up ceph-mgr-k8sevents (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:43.023 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:43.038 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T23:47:43.038 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T23:47:43.149 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up ceph-mon (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:43.353 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T23:47:43.353 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T23:47:43.538 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up libcephfs-dev (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:43.656 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-rados (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:43.765 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up librgw2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:43.834 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up ceph-mgr-cephadm (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:43.892 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T23:47:43.985 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up ceph (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:44.111 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up ceph-mgr-dashboard (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:44.254 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up ceph-volume (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:44.655 INFO:teuthology.orchestra.run.smithi080.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T23:47:44.807 INFO:teuthology.orchestra.run.smithi080.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T23:47:44.952 INFO:teuthology.orchestra.run.smithi080.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T23:47:45.229 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-rbd (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:45.330 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up rbd-fuse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:45.456 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-rgw (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:45.590 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-cephfs (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:45.716 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up ceph-common (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:46.020 INFO:teuthology.orchestra.run.smithi080.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T23:47:46.259 INFO:teuthology.orchestra.run.smithi057.stdout:Adding group ceph....done 2024-04-25T23:47:46.636 INFO:teuthology.orchestra.run.smithi057.stdout:Adding system user ceph....done 2024-04-25T23:47:46.822 INFO:teuthology.orchestra.run.smithi057.stdout:Setting system user ceph properties....done 2024-04-25T23:47:46.840 INFO:teuthology.orchestra.run.smithi057.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-25T23:47:46.982 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-25T23:47:47.118 INFO:teuthology.orchestra.run.smithi080.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T23:47:47.121 DEBUG:teuthology.orchestra.run.smithi080:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install sqlite3 2024-04-25T23:47:47.220 INFO:teuthology.orchestra.run.smithi080.stdout:Reading package lists... 2024-04-25T23:47:47.293 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-25T23:47:47.424 INFO:teuthology.orchestra.run.smithi080.stdout:Building dependency tree... 2024-04-25T23:47:47.424 INFO:teuthology.orchestra.run.smithi080.stdout:Reading state information... 2024-04-25T23:47:47.604 INFO:teuthology.orchestra.run.smithi080.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T23:47:47.605 INFO:teuthology.orchestra.run.smithi080.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T23:47:47.605 INFO:teuthology.orchestra.run.smithi080.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T23:47:47.605 INFO:teuthology.orchestra.run.smithi080.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T23:47:47.605 INFO:teuthology.orchestra.run.smithi080.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T23:47:47.605 INFO:teuthology.orchestra.run.smithi080.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T23:47:47.605 INFO:teuthology.orchestra.run.smithi080.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T23:47:47.606 INFO:teuthology.orchestra.run.smithi080.stdout:The following additional packages will be installed: 2024-04-25T23:47:47.606 INFO:teuthology.orchestra.run.smithi080.stdout: libsqlite3-0 2024-04-25T23:47:47.607 INFO:teuthology.orchestra.run.smithi080.stdout:Suggested packages: 2024-04-25T23:47:47.608 INFO:teuthology.orchestra.run.smithi080.stdout: sqlite3-doc 2024-04-25T23:47:47.634 INFO:teuthology.orchestra.run.smithi080.stdout:The following NEW packages will be installed: 2024-04-25T23:47:47.635 INFO:teuthology.orchestra.run.smithi080.stdout: sqlite3 2024-04-25T23:47:47.636 INFO:teuthology.orchestra.run.smithi080.stdout:The following packages will be upgraded: 2024-04-25T23:47:47.636 INFO:teuthology.orchestra.run.smithi080.stdout: libsqlite3-0 2024-04-25T23:47:47.693 INFO:teuthology.orchestra.run.smithi080.stdout:1 upgraded, 1 newly installed, 0 to remove and 301 not upgraded. 2024-04-25T23:47:47.694 INFO:teuthology.orchestra.run.smithi080.stdout:Need to get 1,410 kB of archives. 2024-04-25T23:47:47.694 INFO:teuthology.orchestra.run.smithi080.stdout:After this operation, 1,873 kB of additional disk space will be used. 2024-04-25T23:47:47.694 INFO:teuthology.orchestra.run.smithi080.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libsqlite3-0 amd64 3.37.2-2ubuntu0.3 [641 kB] 2024-04-25T23:47:47.841 INFO:teuthology.orchestra.run.smithi080.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 sqlite3 amd64 3.37.2-2ubuntu0.3 [768 kB] 2024-04-25T23:47:47.961 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up ceph-test (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:48.087 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up radosgw (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:48.191 INFO:teuthology.orchestra.run.smithi080.stdout:Fetched 1,410 kB in 0s (6,102 kB/s) 2024-04-25T23:47:48.468 INFO:teuthology.orchestra.run.smithi080.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173891 files and directories currently installed.) 2024-04-25T23:47:48.474 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../libsqlite3-0_3.37.2-2ubuntu0.3_amd64.deb ... 2024-04-25T23:47:48.529 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T23:47:48.529 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T23:47:48.633 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libsqlite3-0:amd64 (3.37.2-2ubuntu0.3) over (3.37.2-2ubuntu0.1) ... 2024-04-25T23:47:48.952 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package sqlite3. 2024-04-25T23:47:48.980 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../sqlite3_3.37.2-2ubuntu0.3_amd64.deb ... 2024-04-25T23:47:49.014 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up ceph-base (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:49.018 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking sqlite3 (3.37.2-2ubuntu0.3) ... 2024-04-25T23:47:49.370 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libsqlite3-0:amd64 (3.37.2-2ubuntu0.3) ... 2024-04-25T23:47:49.380 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-25T23:47:49.497 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up sqlite3 (3.37.2-2ubuntu0.3) ... 2024-04-25T23:47:49.616 INFO:teuthology.orchestra.run.smithi080.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T23:47:49.883 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up ceph-mds (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:50.124 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T23:47:50.125 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T23:47:50.610 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up ceph-mgr (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:50.692 INFO:teuthology.orchestra.run.smithi080.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T23:47:50.836 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T23:47:50.836 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T23:47:51.164 INFO:teuthology.orchestra.run.smithi080.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T23:47:51.170 DEBUG:teuthology.parallel:result is None 2024-04-25T23:47:51.328 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up ceph-osd (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:51.606 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T23:47:51.607 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T23:47:52.342 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up ceph-mgr-k8sevents (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:52.443 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:52.594 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up ceph-mon (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:52.815 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T23:47:52.816 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T23:47:53.346 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up ceph-mgr-cephadm (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:53.522 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up ceph (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:53.697 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up ceph-mgr-dashboard (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:53.908 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up ceph-volume (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T23:47:54.278 INFO:teuthology.orchestra.run.smithi057.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T23:47:54.378 INFO:teuthology.orchestra.run.smithi057.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T23:47:54.547 INFO:teuthology.orchestra.run.smithi057.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T23:47:55.656 INFO:teuthology.orchestra.run.smithi057.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T23:47:56.313 INFO:teuthology.orchestra.run.smithi057.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T23:47:56.316 DEBUG:teuthology.orchestra.run.smithi057:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install sqlite3 2024-04-25T23:47:56.370 INFO:teuthology.orchestra.run.smithi057.stdout:Reading package lists... 2024-04-25T23:47:56.574 INFO:teuthology.orchestra.run.smithi057.stdout:Building dependency tree... 2024-04-25T23:47:56.575 INFO:teuthology.orchestra.run.smithi057.stdout:Reading state information... 2024-04-25T23:47:56.769 INFO:teuthology.orchestra.run.smithi057.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T23:47:56.769 INFO:teuthology.orchestra.run.smithi057.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T23:47:56.769 INFO:teuthology.orchestra.run.smithi057.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T23:47:56.769 INFO:teuthology.orchestra.run.smithi057.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T23:47:56.770 INFO:teuthology.orchestra.run.smithi057.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T23:47:56.770 INFO:teuthology.orchestra.run.smithi057.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T23:47:56.770 INFO:teuthology.orchestra.run.smithi057.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T23:47:56.771 INFO:teuthology.orchestra.run.smithi057.stdout:The following additional packages will be installed: 2024-04-25T23:47:56.771 INFO:teuthology.orchestra.run.smithi057.stdout: libsqlite3-0 2024-04-25T23:47:56.773 INFO:teuthology.orchestra.run.smithi057.stdout:Suggested packages: 2024-04-25T23:47:56.773 INFO:teuthology.orchestra.run.smithi057.stdout: sqlite3-doc 2024-04-25T23:47:56.803 INFO:teuthology.orchestra.run.smithi057.stdout:The following NEW packages will be installed: 2024-04-25T23:47:56.803 INFO:teuthology.orchestra.run.smithi057.stdout: sqlite3 2024-04-25T23:47:56.804 INFO:teuthology.orchestra.run.smithi057.stdout:The following packages will be upgraded: 2024-04-25T23:47:56.804 INFO:teuthology.orchestra.run.smithi057.stdout: libsqlite3-0 2024-04-25T23:47:56.862 INFO:teuthology.orchestra.run.smithi057.stdout:1 upgraded, 1 newly installed, 0 to remove and 301 not upgraded. 2024-04-25T23:47:56.862 INFO:teuthology.orchestra.run.smithi057.stdout:Need to get 1,410 kB of archives. 2024-04-25T23:47:56.862 INFO:teuthology.orchestra.run.smithi057.stdout:After this operation, 1,873 kB of additional disk space will be used. 2024-04-25T23:47:56.862 INFO:teuthology.orchestra.run.smithi057.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libsqlite3-0 amd64 3.37.2-2ubuntu0.3 [641 kB] 2024-04-25T23:47:57.006 INFO:teuthology.orchestra.run.smithi057.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 sqlite3 amd64 3.37.2-2ubuntu0.3 [768 kB] 2024-04-25T23:47:57.399 INFO:teuthology.orchestra.run.smithi057.stdout:Fetched 1,410 kB in 0s (6,417 kB/s) 2024-04-25T23:47:57.687 INFO:teuthology.orchestra.run.smithi057.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173891 files and directories currently installed.) 2024-04-25T23:47:57.693 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../libsqlite3-0_3.37.2-2ubuntu0.3_amd64.deb ... 2024-04-25T23:47:57.837 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking libsqlite3-0:amd64 (3.37.2-2ubuntu0.3) over (3.37.2-2ubuntu0.1) ... 2024-04-25T23:47:58.133 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package sqlite3. 2024-04-25T23:47:58.150 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../sqlite3_3.37.2-2ubuntu0.3_amd64.deb ... 2024-04-25T23:47:58.191 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking sqlite3 (3.37.2-2ubuntu0.3) ... 2024-04-25T23:47:58.544 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up libsqlite3-0:amd64 (3.37.2-2ubuntu0.3) ... 2024-04-25T23:47:58.679 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up sqlite3 (3.37.2-2ubuntu0.3) ... 2024-04-25T23:47:58.806 INFO:teuthology.orchestra.run.smithi057.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T23:47:59.924 INFO:teuthology.orchestra.run.smithi057.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T23:48:00.400 INFO:teuthology.orchestra.run.smithi057.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T23:48:00.404 DEBUG:teuthology.parallel:result is None 2024-04-25T23:48:00.405 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T23:48:00.554 DEBUG:teuthology.orchestra.run.smithi057:> dpkg-query -W -f '${Version}' ceph 2024-04-25T23:48:00.573 INFO:teuthology.orchestra.run.smithi057.stdout:19.0.0-2445-gd2c45d97-1jammy 2024-04-25T23:48:00.574 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2445-gd2c45d97-1jammy 2024-04-25T23:48:00.574 INFO:teuthology.task.install:The correct ceph version 19.0.0-2445-gd2c45d97-1jammy is installed. 2024-04-25T23:48:00.577 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T23:48:00.722 DEBUG:teuthology.orchestra.run.smithi080:> dpkg-query -W -f '${Version}' ceph 2024-04-25T23:48:00.741 INFO:teuthology.orchestra.run.smithi080.stdout:19.0.0-2445-gd2c45d97-1jammy 2024-04-25T23:48:00.742 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2445-gd2c45d97-1jammy 2024-04-25T23:48:00.742 INFO:teuthology.task.install:The correct ceph version 19.0.0-2445-gd2c45d97-1jammy is installed. 2024-04-25T23:48:00.744 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T23:48:00.744 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-04-25T23:48:00.744 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T23:48:00.759 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T23:48:00.759 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T23:48:00.799 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T23:48:00.799 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-04-25T23:48:00.800 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T23:48:00.815 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T23:48:00.870 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T23:48:00.870 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T23:48:00.883 DEBUG:teuthology.orchestra.run.smithi080:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T23:48:00.937 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T23:48:00.938 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-04-25T23:48:00.938 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T23:48:00.951 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T23:48:01.006 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T23:48:01.006 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T23:48:01.019 DEBUG:teuthology.orchestra.run.smithi080:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T23:48:01.074 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T23:48:01.074 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-04-25T23:48:01.074 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T23:48:01.087 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T23:48:01.142 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T23:48:01.142 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T23:48:01.154 DEBUG:teuthology.orchestra.run.smithi080:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T23:48:01.208 INFO:teuthology.run_tasks:Running task ceph... 2024-04-25T23:48:01.295 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-25T23:48:01.295 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod 777 /var/log/ceph 2024-04-25T23:48:01.297 DEBUG:teuthology.orchestra.run.smithi080:> sudo chmod 777 /var/log/ceph 2024-04-25T23:48:01.310 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-25T23:48:01.310 DEBUG:teuthology.orchestra.run.smithi057:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T23:48:01.351 DEBUG:teuthology.orchestra.run.smithi080:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T23:48:01.365 INFO:tasks.ceph:Creating extra log directories... 2024-04-25T23:48:01.365 DEBUG:teuthology.orchestra.run.smithi057:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T23:48:01.407 DEBUG:teuthology.orchestra.run.smithi080:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T23:48:01.429 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-25T23:48:01.429 INFO:tasks.ceph:config {'conf': {'client': {'debug cephsqlite': 20, 'debug client': 20, 'debug ms': 1}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject socket failures': 1500, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-25T23:48:01.430 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-04-25_21:00:03-rados-squid-distro-default-smithi/7673825', 'branch': 'squid', 'description': 'rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} tasks/libcephsqlite}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7673825', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-04-25_21:00:03-rados-squid-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'volumes': {'count': 4, 'size': 10}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'client': {'debug cephsqlite': 20, 'debug client': 20, 'debug ms': 1}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject socket failures': 1500, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled'], 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'b22e2ebdeb24376882b7bda2a7329c8cccc2276a'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 101, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 9886, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'sleep_before_teardown': 0, 'subset': '30270/100000', 'suite': 'rados', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_b22e2ebdeb24376882b7bda2a7329c8cccc2276a/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': 'b22e2ebdeb24376882b7bda2a7329c8cccc2276a', 'targets': {'smithi057.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD6eQPdygOnUDfXan2c16/gwJdK6U1jFxx0VWVcM4g/QWx2r/ZEB85KlUChC3Ybyjvpw+DPEEc53uTveYuQ5jyM=', 'smithi080.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM6Tgsrau0boLdwiy6/PvpPRBdo3WsqpxWOW+nDJOtz8Ev5gKHclznjvnCLL8KUm3vwA6SUZLB83ZqlgbG25tH0='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_system_packages': {'deb': ['sqlite3'], 'rpm': ['sqlite-devel']}, 'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}}, {'ceph': None}, {'exec': {'client.0': ['ceph osd pool create cephsqlite', "ceph auth get-or-create client.libcephsqlite mon 'profile simple-rados-client-with-blocklist' osd 'allow rwx pool=cephsqlite' >> /etc/ceph/ceph.keyring"]}}, {'exec': {'client.0': ['ceph_test_libcephsqlite --id libcephsqlite --no-log-to-stderr']}}, {'workunit': {'clients': {'client.0': ['rados/test_libcephsqlite.sh cephsqlite']}, 'env': {'CEPH_ARGS': '--id libcephsqlite --no-log-to-stderr'}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-25_21:00:03', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-25T23:48:01.430 DEBUG:teuthology.orchestra.run.smithi057:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T23:48:01.463 DEBUG:teuthology.orchestra.run.smithi080:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T23:48:01.469 DEBUG:teuthology.orchestra.run.smithi057:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T23:48:01.511 DEBUG:teuthology.orchestra.run.smithi080:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T23:48:01.525 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-04-25T23:48:01.525 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T23:48:01.574 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T23:48:01.575 DEBUG:teuthology.orchestra.run.smithi057:> stat /dev/vg_nvme/lv_1 2024-04-25T23:48:01.622 INFO:teuthology.orchestra.run.smithi057.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T23:48:01.623 INFO:teuthology.orchestra.run.smithi057.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T23:48:01.623 INFO:teuthology.orchestra.run.smithi057.stdout:Device: 5h/5d Inode: 852 Links: 1 2024-04-25T23:48:01.623 INFO:teuthology.orchestra.run.smithi057.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T23:48:01.623 INFO:teuthology.orchestra.run.smithi057.stdout:Access: 2024-04-25 23:42:09.505186311 +0000 2024-04-25T23:48:01.623 INFO:teuthology.orchestra.run.smithi057.stdout:Modify: 2024-04-25 23:42:09.501186346 +0000 2024-04-25T23:48:01.623 INFO:teuthology.orchestra.run.smithi057.stdout:Change: 2024-04-25 23:42:09.501186346 +0000 2024-04-25T23:48:01.623 INFO:teuthology.orchestra.run.smithi057.stdout: Birth: - 2024-04-25T23:48:01.623 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T23:48:01.677 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records in 2024-04-25T23:48:01.677 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records out 2024-04-25T23:48:01.677 INFO:teuthology.orchestra.run.smithi057.stderr:512 bytes copied, 0.000543046 s, 943 kB/s 2024-04-25T23:48:01.678 DEBUG:teuthology.orchestra.run.smithi057:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T23:48:01.728 DEBUG:teuthology.orchestra.run.smithi057:> stat /dev/vg_nvme/lv_2 2024-04-25T23:48:01.779 INFO:teuthology.orchestra.run.smithi057.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T23:48:01.779 INFO:teuthology.orchestra.run.smithi057.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T23:48:01.779 INFO:teuthology.orchestra.run.smithi057.stdout:Device: 5h/5d Inode: 870 Links: 1 2024-04-25T23:48:01.779 INFO:teuthology.orchestra.run.smithi057.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T23:48:01.779 INFO:teuthology.orchestra.run.smithi057.stdout:Access: 2024-04-25 23:42:10.217179951 +0000 2024-04-25T23:48:01.779 INFO:teuthology.orchestra.run.smithi057.stdout:Modify: 2024-04-25 23:42:09.985182024 +0000 2024-04-25T23:48:01.779 INFO:teuthology.orchestra.run.smithi057.stdout:Change: 2024-04-25 23:42:09.985182024 +0000 2024-04-25T23:48:01.779 INFO:teuthology.orchestra.run.smithi057.stdout: Birth: - 2024-04-25T23:48:01.779 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T23:48:01.833 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records in 2024-04-25T23:48:01.833 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records out 2024-04-25T23:48:01.833 INFO:teuthology.orchestra.run.smithi057.stderr:512 bytes copied, 0.000449876 s, 1.1 MB/s 2024-04-25T23:48:01.834 DEBUG:teuthology.orchestra.run.smithi057:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T23:48:01.884 DEBUG:teuthology.orchestra.run.smithi057:> stat /dev/vg_nvme/lv_3 2024-04-25T23:48:01.931 INFO:teuthology.orchestra.run.smithi057.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T23:48:01.931 INFO:teuthology.orchestra.run.smithi057.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T23:48:01.931 INFO:teuthology.orchestra.run.smithi057.stdout:Device: 5h/5d Inode: 887 Links: 1 2024-04-25T23:48:01.931 INFO:teuthology.orchestra.run.smithi057.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T23:48:01.931 INFO:teuthology.orchestra.run.smithi057.stdout:Access: 2024-04-25 23:42:10.665175950 +0000 2024-04-25T23:48:01.931 INFO:teuthology.orchestra.run.smithi057.stdout:Modify: 2024-04-25 23:42:10.421178129 +0000 2024-04-25T23:48:01.931 INFO:teuthology.orchestra.run.smithi057.stdout:Change: 2024-04-25 23:42:10.421178129 +0000 2024-04-25T23:48:01.931 INFO:teuthology.orchestra.run.smithi057.stdout: Birth: - 2024-04-25T23:48:01.931 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T23:48:01.985 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records in 2024-04-25T23:48:01.985 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records out 2024-04-25T23:48:01.985 INFO:teuthology.orchestra.run.smithi057.stderr:512 bytes copied, 0.000563399 s, 909 kB/s 2024-04-25T23:48:01.987 DEBUG:teuthology.orchestra.run.smithi057:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T23:48:02.036 DEBUG:teuthology.orchestra.run.smithi057:> stat /dev/vg_nvme/lv_4 2024-04-25T23:48:02.086 INFO:teuthology.orchestra.run.smithi057.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T23:48:02.087 INFO:teuthology.orchestra.run.smithi057.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T23:48:02.087 INFO:teuthology.orchestra.run.smithi057.stdout:Device: 5h/5d Inode: 905 Links: 1 2024-04-25T23:48:02.087 INFO:teuthology.orchestra.run.smithi057.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T23:48:02.087 INFO:teuthology.orchestra.run.smithi057.stdout:Access: 2024-04-25 23:42:11.161171520 +0000 2024-04-25T23:48:02.087 INFO:teuthology.orchestra.run.smithi057.stdout:Modify: 2024-04-25 23:42:10.921173663 +0000 2024-04-25T23:48:02.087 INFO:teuthology.orchestra.run.smithi057.stdout:Change: 2024-04-25 23:42:10.921173663 +0000 2024-04-25T23:48:02.087 INFO:teuthology.orchestra.run.smithi057.stdout: Birth: - 2024-04-25T23:48:02.087 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T23:48:02.141 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records in 2024-04-25T23:48:02.141 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records out 2024-04-25T23:48:02.141 INFO:teuthology.orchestra.run.smithi057.stderr:512 bytes copied, 0.00051817 s, 988 kB/s 2024-04-25T23:48:02.143 DEBUG:teuthology.orchestra.run.smithi057:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T23:48:02.192 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T23:48:02.192 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T23:48:02.193 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T23:48:02.199 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T23:48:02.199 DEBUG:teuthology.orchestra.run.smithi080:> stat /dev/vg_nvme/lv_1 2024-04-25T23:48:02.247 INFO:teuthology.orchestra.run.smithi080.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T23:48:02.247 INFO:teuthology.orchestra.run.smithi080.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T23:48:02.248 INFO:teuthology.orchestra.run.smithi080.stdout:Device: 5h/5d Inode: 878 Links: 1 2024-04-25T23:48:02.248 INFO:teuthology.orchestra.run.smithi080.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T23:48:02.248 INFO:teuthology.orchestra.run.smithi080.stdout:Access: 2024-04-25 23:42:05.544413319 +0000 2024-04-25T23:48:02.248 INFO:teuthology.orchestra.run.smithi080.stdout:Modify: 2024-04-25 23:42:05.256415892 +0000 2024-04-25T23:48:02.248 INFO:teuthology.orchestra.run.smithi080.stdout:Change: 2024-04-25 23:42:05.256415892 +0000 2024-04-25T23:48:02.248 INFO:teuthology.orchestra.run.smithi080.stdout: Birth: - 2024-04-25T23:48:02.249 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T23:48:02.301 INFO:teuthology.orchestra.run.smithi080.stderr:1+0 records in 2024-04-25T23:48:02.301 INFO:teuthology.orchestra.run.smithi080.stderr:1+0 records out 2024-04-25T23:48:02.301 INFO:teuthology.orchestra.run.smithi080.stderr:512 bytes copied, 0.000412911 s, 1.2 MB/s 2024-04-25T23:48:02.303 DEBUG:teuthology.orchestra.run.smithi080:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T23:48:02.351 DEBUG:teuthology.orchestra.run.smithi080:> stat /dev/vg_nvme/lv_2 2024-04-25T23:48:02.399 INFO:teuthology.orchestra.run.smithi080.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T23:48:02.399 INFO:teuthology.orchestra.run.smithi080.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T23:48:02.399 INFO:teuthology.orchestra.run.smithi080.stdout:Device: 5h/5d Inode: 889 Links: 1 2024-04-25T23:48:02.399 INFO:teuthology.orchestra.run.smithi080.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T23:48:02.399 INFO:teuthology.orchestra.run.smithi080.stdout:Access: 2024-04-25 23:42:06.012409138 +0000 2024-04-25T23:48:02.399 INFO:teuthology.orchestra.run.smithi080.stdout:Modify: 2024-04-25 23:42:05.764411354 +0000 2024-04-25T23:48:02.399 INFO:teuthology.orchestra.run.smithi080.stdout:Change: 2024-04-25 23:42:05.764411354 +0000 2024-04-25T23:48:02.399 INFO:teuthology.orchestra.run.smithi080.stdout: Birth: - 2024-04-25T23:48:02.400 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T23:48:02.452 INFO:teuthology.orchestra.run.smithi080.stderr:1+0 records in 2024-04-25T23:48:02.453 INFO:teuthology.orchestra.run.smithi080.stderr:1+0 records out 2024-04-25T23:48:02.453 INFO:teuthology.orchestra.run.smithi080.stderr:512 bytes copied, 0.000416526 s, 1.2 MB/s 2024-04-25T23:48:02.454 DEBUG:teuthology.orchestra.run.smithi080:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T23:48:02.503 DEBUG:teuthology.orchestra.run.smithi080:> stat /dev/vg_nvme/lv_3 2024-04-25T23:48:02.551 INFO:teuthology.orchestra.run.smithi080.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T23:48:02.551 INFO:teuthology.orchestra.run.smithi080.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T23:48:02.551 INFO:teuthology.orchestra.run.smithi080.stdout:Device: 5h/5d Inode: 906 Links: 1 2024-04-25T23:48:02.551 INFO:teuthology.orchestra.run.smithi080.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T23:48:02.551 INFO:teuthology.orchestra.run.smithi080.stdout:Access: 2024-04-25 23:42:06.252406995 +0000 2024-04-25T23:48:02.552 INFO:teuthology.orchestra.run.smithi080.stdout:Modify: 2024-04-25 23:42:06.248407030 +0000 2024-04-25T23:48:02.552 INFO:teuthology.orchestra.run.smithi080.stdout:Change: 2024-04-25 23:42:06.248407030 +0000 2024-04-25T23:48:02.552 INFO:teuthology.orchestra.run.smithi080.stdout: Birth: - 2024-04-25T23:48:02.552 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T23:48:02.605 INFO:teuthology.orchestra.run.smithi080.stderr:1+0 records in 2024-04-25T23:48:02.605 INFO:teuthology.orchestra.run.smithi080.stderr:1+0 records out 2024-04-25T23:48:02.605 INFO:teuthology.orchestra.run.smithi080.stderr:512 bytes copied, 0.000365927 s, 1.4 MB/s 2024-04-25T23:48:02.606 DEBUG:teuthology.orchestra.run.smithi080:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T23:48:02.655 DEBUG:teuthology.orchestra.run.smithi080:> stat /dev/vg_nvme/lv_4 2024-04-25T23:48:02.702 INFO:teuthology.orchestra.run.smithi080.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T23:48:02.702 INFO:teuthology.orchestra.run.smithi080.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T23:48:02.702 INFO:teuthology.orchestra.run.smithi080.stdout:Device: 5h/5d Inode: 931 Links: 1 2024-04-25T23:48:02.702 INFO:teuthology.orchestra.run.smithi080.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T23:48:02.702 INFO:teuthology.orchestra.run.smithi080.stdout:Access: 2024-04-25 23:42:06.984400457 +0000 2024-04-25T23:48:02.702 INFO:teuthology.orchestra.run.smithi080.stdout:Modify: 2024-04-25 23:42:06.728402743 +0000 2024-04-25T23:48:02.702 INFO:teuthology.orchestra.run.smithi080.stdout:Change: 2024-04-25 23:42:06.728402743 +0000 2024-04-25T23:48:02.703 INFO:teuthology.orchestra.run.smithi080.stdout: Birth: - 2024-04-25T23:48:02.703 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T23:48:02.756 INFO:teuthology.orchestra.run.smithi080.stderr:1+0 records in 2024-04-25T23:48:02.756 INFO:teuthology.orchestra.run.smithi080.stderr:1+0 records out 2024-04-25T23:48:02.756 INFO:teuthology.orchestra.run.smithi080.stderr:512 bytes copied, 0.00038278 s, 1.3 MB/s 2024-04-25T23:48:02.757 DEBUG:teuthology.orchestra.run.smithi080:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T23:48:02.807 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T23:48:02.807 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi057.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi080.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T23:48:02.807 INFO:tasks.ceph:Generating config... 2024-04-25T23:48:02.808 INFO:tasks.ceph:[client] debug cephsqlite = 20 2024-04-25T23:48:02.809 INFO:tasks.ceph:[client] debug client = 20 2024-04-25T23:48:02.809 INFO:tasks.ceph:[client] debug ms = 1 2024-04-25T23:48:02.809 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-04-25T23:48:02.809 INFO:tasks.ceph:[global] mon election default strategy = 3 2024-04-25T23:48:02.809 INFO:tasks.ceph:[global] ms bind msgr1 = False 2024-04-25T23:48:02.809 INFO:tasks.ceph:[global] ms bind msgr2 = True 2024-04-25T23:48:02.809 INFO:tasks.ceph:[global] ms inject socket failures = 1500 2024-04-25T23:48:02.809 INFO:tasks.ceph:[global] ms type = async 2024-04-25T23:48:02.809 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-25T23:48:02.809 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-25T23:48:02.809 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-25T23:48:02.809 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-25T23:48:02.809 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-25T23:48:02.809 INFO:tasks.ceph:[mon] mon min osdmap epochs = 50 2024-04-25T23:48:02.809 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2024-04-25T23:48:02.810 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2024-04-25T23:48:02.810 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2024-04-25T23:48:02.810 INFO:tasks.ceph:[mon] mon scrub interval = 300 2024-04-25T23:48:02.810 INFO:tasks.ceph:[mon] paxos service trim min = 10 2024-04-25T23:48:02.810 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-25T23:48:02.810 INFO:tasks.ceph:[osd] bluestore compression algorithm = zstd 2024-04-25T23:48:02.810 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-04-25T23:48:02.810 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-25T23:48:02.810 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2024-04-25T23:48:02.810 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-04-25T23:48:02.810 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-04-25T23:48:02.810 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-25T23:48:02.810 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-25T23:48:02.810 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-04-25T23:48:02.811 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-25T23:48:02.811 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-25T23:48:02.811 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-25T23:48:02.811 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2024-04-25T23:48:02.811 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2024-04-25T23:48:02.811 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-25T23:48:02.811 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2024-04-25T23:48:02.811 INFO:tasks.ceph:[osd] osd mclock profile = high_recovery_ops 2024-04-25T23:48:02.811 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-25T23:48:02.811 INFO:tasks.ceph:[osd] osd op queue = debug_random 2024-04-25T23:48:02.811 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2024-04-25T23:48:02.811 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-04-25T23:48:02.811 INFO:tasks.ceph:Setting up mon.a... 2024-04-25T23:48:02.812 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-25T23:48:02.847 INFO:teuthology.orchestra.run.smithi057.stdout:creating /etc/ceph/ceph.keyring 2024-04-25T23:48:02.858 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-25T23:48:02.941 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T23:48:02.998 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.57'), ('mon.c', '[v2:172.21.15.57:3301,v1:172.21.15.57:6790]'), ('mon.b', '172.21.15.80')] 2024-04-25T23:48:02.998 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.57,[v2:172.21.15.57:3301,v1:172.21.15.57:6790],172.21.15.80', 'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject socket failures': 1500, 'ms type': 'async'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug cephsqlite': 20, 'debug client': 20, 'debug ms': 1}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-04-25T23:48:02.999 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-04-25T23:48:02.999 DEBUG:teuthology.orchestra.run.smithi057:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T23:48:03.046 DEBUG:teuthology.orchestra.run.smithi057:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.57 --addv c '[v2:172.21.15.57:3301,v1:172.21.15.57:6790]' --add b 172.21.15.80 --print /home/ubuntu/cephtest/ceph.monmap 2024-04-25T23:48:03.133 INFO:teuthology.orchestra.run.smithi057.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-25T23:48:03.133 INFO:teuthology.orchestra.run.smithi057.stdout:monmaptool: generated fsid ca0adda3-98dd-4ba5-b4b2-8b4f20dcabdd 2024-04-25T23:48:03.133 INFO:teuthology.orchestra.run.smithi057.stdout:setting min_mon_release = quincy 2024-04-25T23:48:03.133 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 0 2024-04-25T23:48:03.134 INFO:teuthology.orchestra.run.smithi057.stdout:fsid ca0adda3-98dd-4ba5-b4b2-8b4f20dcabdd 2024-04-25T23:48:03.134 INFO:teuthology.orchestra.run.smithi057.stdout:last_changed 2024-04-25T23:48:03.150030+0000 2024-04-25T23:48:03.134 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-04-25T23:48:03.150030+0000 2024-04-25T23:48:03.134 INFO:teuthology.orchestra.run.smithi057.stdout:min_mon_release 17 (quincy) 2024-04-25T23:48:03.134 INFO:teuthology.orchestra.run.smithi057.stdout:election_strategy: 3 2024-04-25T23:48:03.134 INFO:teuthology.orchestra.run.smithi057.stdout:0: [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] mon.a 2024-04-25T23:48:03.134 INFO:teuthology.orchestra.run.smithi057.stdout:1: [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] mon.b 2024-04-25T23:48:03.134 INFO:teuthology.orchestra.run.smithi057.stdout:2: [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] mon.c 2024-04-25T23:48:03.134 INFO:teuthology.orchestra.run.smithi057.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-25T23:48:03.135 DEBUG:teuthology.orchestra.run.smithi057:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T23:48:03.182 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID ca0adda3-98dd-4ba5-b4b2-8b4f20dcabdd... 2024-04-25T23:48:03.183 DEBUG:teuthology.orchestra.run.smithi057:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T23:48:03.227 DEBUG:teuthology.orchestra.run.smithi080:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T23:48:03.256 INFO:teuthology.orchestra.run.smithi057.stdout:[global] 2024-04-25T23:48:03.256 INFO:teuthology.orchestra.run.smithi057.stdout: chdir = "" 2024-04-25T23:48:03.256 INFO:teuthology.orchestra.run.smithi057.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T23:48:03.256 INFO:teuthology.orchestra.run.smithi057.stdout: auth supported = cephx 2024-04-25T23:48:03.256 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-25T23:48:03.256 INFO:teuthology.orchestra.run.smithi057.stdout: filestore xattr use omap = true 2024-04-25T23:48:03.256 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-25T23:48:03.256 INFO:teuthology.orchestra.run.smithi057.stdout: mon clock drift allowed = 1.000 2024-04-25T23:48:03.256 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-25T23:48:03.256 INFO:teuthology.orchestra.run.smithi057.stdout: osd crush chooseleaf type = 0 2024-04-25T23:48:03.257 INFO:teuthology.orchestra.run.smithi057.stdout: auth debug = true 2024-04-25T23:48:03.257 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-25T23:48:03.257 INFO:teuthology.orchestra.run.smithi057.stdout: ms die on old message = true 2024-04-25T23:48:03.257 INFO:teuthology.orchestra.run.smithi057.stdout: ms die on bug = true 2024-04-25T23:48:03.257 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-25T23:48:03.257 INFO:teuthology.orchestra.run.smithi057.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T23:48:03.257 INFO:teuthology.orchestra.run.smithi057.stdout: mon pg warn max object skew = 0 2024-04-25T23:48:03.257 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-25T23:48:03.257 INFO:teuthology.orchestra.run.smithi057.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T23:48:03.257 INFO:teuthology.orchestra.run.smithi057.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T23:48:03.257 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-25T23:48:03.257 INFO:teuthology.orchestra.run.smithi057.stdout: osd pool default size = 2 2024-04-25T23:48:03.257 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-25T23:48:03.257 INFO:teuthology.orchestra.run.smithi057.stdout: mon osd allow primary affinity = true 2024-04-25T23:48:03.257 INFO:teuthology.orchestra.run.smithi057.stdout: mon osd allow pg remap = true 2024-04-25T23:48:03.258 INFO:teuthology.orchestra.run.smithi057.stdout: mon warn on legacy crush tunables = false 2024-04-25T23:48:03.258 INFO:teuthology.orchestra.run.smithi057.stdout: mon warn on crush straw calc version zero = false 2024-04-25T23:48:03.258 INFO:teuthology.orchestra.run.smithi057.stdout: mon warn on no sortbitwise = false 2024-04-25T23:48:03.258 INFO:teuthology.orchestra.run.smithi057.stdout: mon warn on osd down out interval zero = false 2024-04-25T23:48:03.258 INFO:teuthology.orchestra.run.smithi057.stdout: mon warn on too few osds = false 2024-04-25T23:48:03.258 INFO:teuthology.orchestra.run.smithi057.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T23:48:03.258 INFO:teuthology.orchestra.run.smithi057.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T23:48:03.258 INFO:teuthology.orchestra.run.smithi057.stdout: mon_allow_pool_size_one = true 2024-04-25T23:48:03.258 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-25T23:48:03.258 INFO:teuthology.orchestra.run.smithi057.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T23:48:03.258 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-25T23:48:03.258 INFO:teuthology.orchestra.run.smithi057.stdout: osd default data pool replay window = 5 2024-04-25T23:48:03.258 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-25T23:48:03.258 INFO:teuthology.orchestra.run.smithi057.stdout: mon allow pool delete = true 2024-04-25T23:48:03.258 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-25T23:48:03.259 INFO:teuthology.orchestra.run.smithi057.stdout: mon cluster log file level = debug 2024-04-25T23:48:03.259 INFO:teuthology.orchestra.run.smithi057.stdout: debug asserts on shutdown = true 2024-04-25T23:48:03.259 INFO:teuthology.orchestra.run.smithi057.stdout: mon health detail to clog = false 2024-04-25T23:48:03.259 INFO:teuthology.orchestra.run.smithi057.stdout: mon host = "172.21.15.57,[v2:172.21.15.57:3301,v1:172.21.15.57:6790],172.21.15.80" 2024-04-25T23:48:03.259 INFO:teuthology.orchestra.run.smithi057.stdout: mon client directed command retry = 5 2024-04-25T23:48:03.259 INFO:teuthology.orchestra.run.smithi057.stdout: mon election default strategy = 3 2024-04-25T23:48:03.259 INFO:teuthology.orchestra.run.smithi057.stdout: ms bind msgr1 = False 2024-04-25T23:48:03.259 INFO:teuthology.orchestra.run.smithi057.stdout: ms bind msgr2 = True 2024-04-25T23:48:03.259 INFO:teuthology.orchestra.run.smithi057.stdout: ms inject socket failures = 1500 2024-04-25T23:48:03.259 INFO:teuthology.orchestra.run.smithi057.stdout: ms type = async 2024-04-25T23:48:03.259 INFO:teuthology.orchestra.run.smithi057.stdout: fsid = ca0adda3-98dd-4ba5-b4b2-8b4f20dcabdd 2024-04-25T23:48:03.259 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-25T23:48:03.259 INFO:teuthology.orchestra.run.smithi057.stdout:[osd] 2024-04-25T23:48:03.259 INFO:teuthology.orchestra.run.smithi057.stdout: osd journal size = 100 2024-04-25T23:48:03.259 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-25T23:48:03.259 INFO:teuthology.orchestra.run.smithi057.stdout: osd scrub load threshold = 5.0 2024-04-25T23:48:03.260 INFO:teuthology.orchestra.run.smithi057.stdout: osd scrub max interval = 600 2024-04-25T23:48:03.260 INFO:teuthology.orchestra.run.smithi057.stdout: osd mclock profile = high_recovery_ops 2024-04-25T23:48:03.260 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-25T23:48:03.260 INFO:teuthology.orchestra.run.smithi057.stdout: osd recover clone overlap = true 2024-04-25T23:48:03.260 INFO:teuthology.orchestra.run.smithi057.stdout: osd recovery max chunk = 1048576 2024-04-25T23:48:03.260 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-25T23:48:03.260 INFO:teuthology.orchestra.run.smithi057.stdout: osd debug shutdown = true 2024-04-25T23:48:03.260 INFO:teuthology.orchestra.run.smithi057.stdout: osd debug op order = true 2024-04-25T23:48:03.260 INFO:teuthology.orchestra.run.smithi057.stdout: osd debug verify stray on activate = true 2024-04-25T23:48:03.260 INFO:teuthology.orchestra.run.smithi057.stdout: osd debug trim objects = true 2024-04-25T23:48:03.260 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-25T23:48:03.260 INFO:teuthology.orchestra.run.smithi057.stdout: osd open classes on start = true 2024-04-25T23:48:03.260 INFO:teuthology.orchestra.run.smithi057.stdout: osd debug pg log writeout = true 2024-04-25T23:48:03.260 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-25T23:48:03.260 INFO:teuthology.orchestra.run.smithi057.stdout: osd deep scrub update digest min age = 30 2024-04-25T23:48:03.260 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-25T23:48:03.261 INFO:teuthology.orchestra.run.smithi057.stdout: osd map max advance = 10 2024-04-25T23:48:03.261 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-25T23:48:03.261 INFO:teuthology.orchestra.run.smithi057.stdout: journal zero on create = true 2024-04-25T23:48:03.261 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-25T23:48:03.261 INFO:teuthology.orchestra.run.smithi057.stdout: filestore ondisk finisher threads = 3 2024-04-25T23:48:03.261 INFO:teuthology.orchestra.run.smithi057.stdout: filestore apply finisher threads = 3 2024-04-25T23:48:03.261 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-25T23:48:03.261 INFO:teuthology.orchestra.run.smithi057.stdout: bdev debug aio = true 2024-04-25T23:48:03.261 INFO:teuthology.orchestra.run.smithi057.stdout: osd debug misdirected ops = true 2024-04-25T23:48:03.261 INFO:teuthology.orchestra.run.smithi057.stdout: bluestore block size = 96636764160 2024-04-25T23:48:03.261 INFO:teuthology.orchestra.run.smithi057.stdout: bluestore compression algorithm = zstd 2024-04-25T23:48:03.261 INFO:teuthology.orchestra.run.smithi057.stdout: bluestore compression mode = aggressive 2024-04-25T23:48:03.261 INFO:teuthology.orchestra.run.smithi057.stdout: bluestore fsck on mount = True 2024-04-25T23:48:03.261 INFO:teuthology.orchestra.run.smithi057.stdout: bluestore zero block detection = True 2024-04-25T23:48:03.261 INFO:teuthology.orchestra.run.smithi057.stdout: debug bluefs = 20 2024-04-25T23:48:03.262 INFO:teuthology.orchestra.run.smithi057.stdout: debug bluestore = 20 2024-04-25T23:48:03.262 INFO:teuthology.orchestra.run.smithi057.stdout: debug ms = 1 2024-04-25T23:48:03.262 INFO:teuthology.orchestra.run.smithi057.stdout: debug osd = 20 2024-04-25T23:48:03.262 INFO:teuthology.orchestra.run.smithi057.stdout: debug rocksdb = 10 2024-04-25T23:48:03.262 INFO:teuthology.orchestra.run.smithi057.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T23:48:03.262 INFO:teuthology.orchestra.run.smithi057.stdout: mon osd full ratio = 0.9 2024-04-25T23:48:03.262 INFO:teuthology.orchestra.run.smithi057.stdout: mon osd nearfull ratio = 0.8 2024-04-25T23:48:03.262 INFO:teuthology.orchestra.run.smithi057.stdout: osd debug verify cached snaps = True 2024-04-25T23:48:03.262 INFO:teuthology.orchestra.run.smithi057.stdout: osd debug verify missing on start = True 2024-04-25T23:48:03.262 INFO:teuthology.orchestra.run.smithi057.stdout: osd failsafe full ratio = 0.95 2024-04-25T23:48:03.262 INFO:teuthology.orchestra.run.smithi057.stdout: osd mclock override recovery settings = True 2024-04-25T23:48:03.262 INFO:teuthology.orchestra.run.smithi057.stdout: osd objectstore = bluestore 2024-04-25T23:48:03.262 INFO:teuthology.orchestra.run.smithi057.stdout: osd op queue = debug_random 2024-04-25T23:48:03.262 INFO:teuthology.orchestra.run.smithi057.stdout: osd op queue cut off = debug_random 2024-04-25T23:48:03.262 INFO:teuthology.orchestra.run.smithi057.stdout: osd shutdown pgref assert = True 2024-04-25T23:48:03.263 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-25T23:48:03.263 INFO:teuthology.orchestra.run.smithi057.stdout:[mgr] 2024-04-25T23:48:03.263 INFO:teuthology.orchestra.run.smithi057.stdout: debug ms = 1 2024-04-25T23:48:03.263 INFO:teuthology.orchestra.run.smithi057.stdout: debug mgr = 20 2024-04-25T23:48:03.263 INFO:teuthology.orchestra.run.smithi057.stdout: debug mon = 20 2024-04-25T23:48:03.263 INFO:teuthology.orchestra.run.smithi057.stdout: debug auth = 20 2024-04-25T23:48:03.263 INFO:teuthology.orchestra.run.smithi057.stdout: mon reweight min pgs per osd = 4 2024-04-25T23:48:03.263 INFO:teuthology.orchestra.run.smithi057.stdout: mon reweight min bytes per osd = 10 2024-04-25T23:48:03.263 INFO:teuthology.orchestra.run.smithi057.stdout: mgr/telemetry/nag = false 2024-04-25T23:48:03.263 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-25T23:48:03.263 INFO:teuthology.orchestra.run.smithi057.stdout:[mon] 2024-04-25T23:48:03.263 INFO:teuthology.orchestra.run.smithi057.stdout: debug ms = 1 2024-04-25T23:48:03.263 INFO:teuthology.orchestra.run.smithi057.stdout: debug mon = 20 2024-04-25T23:48:03.263 INFO:teuthology.orchestra.run.smithi057.stdout: debug paxos = 20 2024-04-25T23:48:03.263 INFO:teuthology.orchestra.run.smithi057.stdout: debug auth = 20 2024-04-25T23:48:03.264 INFO:teuthology.orchestra.run.smithi057.stdout: mon data avail warn = 5 2024-04-25T23:48:03.264 INFO:teuthology.orchestra.run.smithi057.stdout: mon mgr mkfs grace = 240 2024-04-25T23:48:03.264 INFO:teuthology.orchestra.run.smithi057.stdout: mon reweight min pgs per osd = 4 2024-04-25T23:48:03.264 INFO:teuthology.orchestra.run.smithi057.stdout: mon osd reporter subtree level = osd 2024-04-25T23:48:03.264 INFO:teuthology.orchestra.run.smithi057.stdout: mon osd prime pg temp = true 2024-04-25T23:48:03.264 INFO:teuthology.orchestra.run.smithi057.stdout: mon reweight min bytes per osd = 10 2024-04-25T23:48:03.264 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-25T23:48:03.264 INFO:teuthology.orchestra.run.smithi057.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T23:48:03.264 INFO:teuthology.orchestra.run.smithi057.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T23:48:03.264 INFO:teuthology.orchestra.run.smithi057.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T23:48:03.264 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-25T23:48:03.264 INFO:teuthology.orchestra.run.smithi057.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T23:48:03.264 INFO:teuthology.orchestra.run.smithi057.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T23:48:03.264 INFO:teuthology.orchestra.run.smithi057.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T23:48:03.264 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-25T23:48:03.264 INFO:teuthology.orchestra.run.smithi057.stdout: # 1m isn't quite enough 2024-04-25T23:48:03.265 INFO:teuthology.orchestra.run.smithi057.stdout: mon_down_mkfs_grace = 2m 2024-04-25T23:48:03.265 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-25T23:48:03.265 INFO:teuthology.orchestra.run.smithi057.stdout: mon_warn_on_filestore_osds = false 2024-04-25T23:48:03.265 INFO:teuthology.orchestra.run.smithi057.stdout: mon min osdmap epochs = 50 2024-04-25T23:48:03.265 INFO:teuthology.orchestra.run.smithi057.stdout: mon osdmap full prune interval = 2 2024-04-25T23:48:03.265 INFO:teuthology.orchestra.run.smithi057.stdout: mon osdmap full prune min = 15 2024-04-25T23:48:03.265 INFO:teuthology.orchestra.run.smithi057.stdout: mon osdmap full prune txsize = 2 2024-04-25T23:48:03.265 INFO:teuthology.orchestra.run.smithi057.stdout: mon scrub interval = 300 2024-04-25T23:48:03.265 INFO:teuthology.orchestra.run.smithi057.stdout: paxos service trim min = 10 2024-04-25T23:48:03.265 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-25T23:48:03.265 INFO:teuthology.orchestra.run.smithi057.stdout:[client] 2024-04-25T23:48:03.265 INFO:teuthology.orchestra.run.smithi057.stdout: rgw cache enabled = true 2024-04-25T23:48:03.265 INFO:teuthology.orchestra.run.smithi057.stdout: rgw enable ops log = true 2024-04-25T23:48:03.265 INFO:teuthology.orchestra.run.smithi057.stdout: rgw enable usage log = true 2024-04-25T23:48:03.265 INFO:teuthology.orchestra.run.smithi057.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T23:48:03.266 INFO:teuthology.orchestra.run.smithi057.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T23:48:03.266 INFO:teuthology.orchestra.run.smithi057.stdout: debug cephsqlite = 20 2024-04-25T23:48:03.266 INFO:teuthology.orchestra.run.smithi057.stdout: debug client = 20 2024-04-25T23:48:03.266 INFO:teuthology.orchestra.run.smithi057.stdout: debug ms = 1 2024-04-25T23:48:03.266 INFO:teuthology.orchestra.run.smithi057.stdout:[mon.a] 2024-04-25T23:48:03.266 INFO:teuthology.orchestra.run.smithi057.stdout:[mon.c] 2024-04-25T23:48:03.266 INFO:teuthology.orchestra.run.smithi057.stdout:[mon.b] 2024-04-25T23:48:03.267 INFO:teuthology.orchestra.run.smithi080.stdout:[global] 2024-04-25T23:48:03.267 INFO:teuthology.orchestra.run.smithi080.stdout: chdir = "" 2024-04-25T23:48:03.267 INFO:teuthology.orchestra.run.smithi080.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T23:48:03.267 INFO:teuthology.orchestra.run.smithi080.stdout: auth supported = cephx 2024-04-25T23:48:03.268 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T23:48:03.268 INFO:teuthology.orchestra.run.smithi080.stdout: filestore xattr use omap = true 2024-04-25T23:48:03.268 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T23:48:03.268 INFO:teuthology.orchestra.run.smithi080.stdout: mon clock drift allowed = 1.000 2024-04-25T23:48:03.268 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T23:48:03.268 INFO:teuthology.orchestra.run.smithi080.stdout: osd crush chooseleaf type = 0 2024-04-25T23:48:03.268 INFO:teuthology.orchestra.run.smithi080.stdout: auth debug = true 2024-04-25T23:48:03.268 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T23:48:03.268 INFO:teuthology.orchestra.run.smithi080.stdout: ms die on old message = true 2024-04-25T23:48:03.268 INFO:teuthology.orchestra.run.smithi080.stdout: ms die on bug = true 2024-04-25T23:48:03.268 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T23:48:03.268 INFO:teuthology.orchestra.run.smithi080.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T23:48:03.268 INFO:teuthology.orchestra.run.smithi080.stdout: mon pg warn max object skew = 0 2024-04-25T23:48:03.268 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T23:48:03.268 INFO:teuthology.orchestra.run.smithi080.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T23:48:03.269 INFO:teuthology.orchestra.run.smithi080.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T23:48:03.269 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T23:48:03.269 INFO:teuthology.orchestra.run.smithi080.stdout: osd pool default size = 2 2024-04-25T23:48:03.269 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T23:48:03.269 INFO:teuthology.orchestra.run.smithi080.stdout: mon osd allow primary affinity = true 2024-04-25T23:48:03.269 INFO:teuthology.orchestra.run.smithi080.stdout: mon osd allow pg remap = true 2024-04-25T23:48:03.269 INFO:teuthology.orchestra.run.smithi080.stdout: mon warn on legacy crush tunables = false 2024-04-25T23:48:03.269 INFO:teuthology.orchestra.run.smithi080.stdout: mon warn on crush straw calc version zero = false 2024-04-25T23:48:03.269 INFO:teuthology.orchestra.run.smithi080.stdout: mon warn on no sortbitwise = false 2024-04-25T23:48:03.269 INFO:teuthology.orchestra.run.smithi080.stdout: mon warn on osd down out interval zero = false 2024-04-25T23:48:03.269 INFO:teuthology.orchestra.run.smithi080.stdout: mon warn on too few osds = false 2024-04-25T23:48:03.269 INFO:teuthology.orchestra.run.smithi080.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T23:48:03.269 INFO:teuthology.orchestra.run.smithi080.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T23:48:03.269 INFO:teuthology.orchestra.run.smithi080.stdout: mon_allow_pool_size_one = true 2024-04-25T23:48:03.269 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T23:48:03.269 INFO:teuthology.orchestra.run.smithi080.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T23:48:03.270 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T23:48:03.270 INFO:teuthology.orchestra.run.smithi080.stdout: osd default data pool replay window = 5 2024-04-25T23:48:03.270 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T23:48:03.270 INFO:teuthology.orchestra.run.smithi080.stdout: mon allow pool delete = true 2024-04-25T23:48:03.270 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T23:48:03.270 INFO:teuthology.orchestra.run.smithi080.stdout: mon cluster log file level = debug 2024-04-25T23:48:03.270 INFO:teuthology.orchestra.run.smithi080.stdout: debug asserts on shutdown = true 2024-04-25T23:48:03.270 INFO:teuthology.orchestra.run.smithi080.stdout: mon health detail to clog = false 2024-04-25T23:48:03.270 INFO:teuthology.orchestra.run.smithi080.stdout: mon host = "172.21.15.57,[v2:172.21.15.57:3301,v1:172.21.15.57:6790],172.21.15.80" 2024-04-25T23:48:03.270 INFO:teuthology.orchestra.run.smithi080.stdout: mon client directed command retry = 5 2024-04-25T23:48:03.270 INFO:teuthology.orchestra.run.smithi080.stdout: mon election default strategy = 3 2024-04-25T23:48:03.270 INFO:teuthology.orchestra.run.smithi080.stdout: ms bind msgr1 = False 2024-04-25T23:48:03.270 INFO:teuthology.orchestra.run.smithi080.stdout: ms bind msgr2 = True 2024-04-25T23:48:03.270 INFO:teuthology.orchestra.run.smithi080.stdout: ms inject socket failures = 1500 2024-04-25T23:48:03.270 INFO:teuthology.orchestra.run.smithi080.stdout: ms type = async 2024-04-25T23:48:03.271 INFO:teuthology.orchestra.run.smithi080.stdout: fsid = ca0adda3-98dd-4ba5-b4b2-8b4f20dcabdd 2024-04-25T23:48:03.271 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T23:48:03.271 INFO:teuthology.orchestra.run.smithi080.stdout:[osd] 2024-04-25T23:48:03.271 INFO:teuthology.orchestra.run.smithi080.stdout: osd journal size = 100 2024-04-25T23:48:03.271 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T23:48:03.271 INFO:teuthology.orchestra.run.smithi080.stdout: osd scrub load threshold = 5.0 2024-04-25T23:48:03.271 INFO:teuthology.orchestra.run.smithi080.stdout: osd scrub max interval = 600 2024-04-25T23:48:03.271 INFO:teuthology.orchestra.run.smithi080.stdout: osd mclock profile = high_recovery_ops 2024-04-25T23:48:03.271 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T23:48:03.271 INFO:teuthology.orchestra.run.smithi080.stdout: osd recover clone overlap = true 2024-04-25T23:48:03.271 INFO:teuthology.orchestra.run.smithi080.stdout: osd recovery max chunk = 1048576 2024-04-25T23:48:03.271 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T23:48:03.271 INFO:teuthology.orchestra.run.smithi080.stdout: osd debug shutdown = true 2024-04-25T23:48:03.271 INFO:teuthology.orchestra.run.smithi080.stdout: osd debug op order = true 2024-04-25T23:48:03.271 INFO:teuthology.orchestra.run.smithi080.stdout: osd debug verify stray on activate = true 2024-04-25T23:48:03.271 INFO:teuthology.orchestra.run.smithi080.stdout: osd debug trim objects = true 2024-04-25T23:48:03.272 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T23:48:03.272 INFO:teuthology.orchestra.run.smithi080.stdout: osd open classes on start = true 2024-04-25T23:48:03.272 INFO:teuthology.orchestra.run.smithi080.stdout: osd debug pg log writeout = true 2024-04-25T23:48:03.272 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T23:48:03.272 INFO:teuthology.orchestra.run.smithi080.stdout: osd deep scrub update digest min age = 30 2024-04-25T23:48:03.272 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T23:48:03.272 INFO:teuthology.orchestra.run.smithi080.stdout: osd map max advance = 10 2024-04-25T23:48:03.272 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T23:48:03.272 INFO:teuthology.orchestra.run.smithi080.stdout: journal zero on create = true 2024-04-25T23:48:03.272 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T23:48:03.272 INFO:teuthology.orchestra.run.smithi080.stdout: filestore ondisk finisher threads = 3 2024-04-25T23:48:03.272 INFO:teuthology.orchestra.run.smithi080.stdout: filestore apply finisher threads = 3 2024-04-25T23:48:03.272 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T23:48:03.272 INFO:teuthology.orchestra.run.smithi080.stdout: bdev debug aio = true 2024-04-25T23:48:03.272 INFO:teuthology.orchestra.run.smithi080.stdout: osd debug misdirected ops = true 2024-04-25T23:48:03.273 INFO:teuthology.orchestra.run.smithi080.stdout: bluestore block size = 96636764160 2024-04-25T23:48:03.273 INFO:teuthology.orchestra.run.smithi080.stdout: bluestore compression algorithm = zstd 2024-04-25T23:48:03.273 INFO:teuthology.orchestra.run.smithi080.stdout: bluestore compression mode = aggressive 2024-04-25T23:48:03.273 INFO:teuthology.orchestra.run.smithi080.stdout: bluestore fsck on mount = True 2024-04-25T23:48:03.273 INFO:teuthology.orchestra.run.smithi080.stdout: bluestore zero block detection = True 2024-04-25T23:48:03.273 INFO:teuthology.orchestra.run.smithi080.stdout: debug bluefs = 20 2024-04-25T23:48:03.273 INFO:teuthology.orchestra.run.smithi080.stdout: debug bluestore = 20 2024-04-25T23:48:03.273 INFO:teuthology.orchestra.run.smithi080.stdout: debug ms = 1 2024-04-25T23:48:03.273 INFO:teuthology.orchestra.run.smithi080.stdout: debug osd = 20 2024-04-25T23:48:03.273 INFO:teuthology.orchestra.run.smithi080.stdout: debug rocksdb = 10 2024-04-25T23:48:03.273 INFO:teuthology.orchestra.run.smithi080.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T23:48:03.273 INFO:teuthology.orchestra.run.smithi080.stdout: mon osd full ratio = 0.9 2024-04-25T23:48:03.273 INFO:teuthology.orchestra.run.smithi080.stdout: mon osd nearfull ratio = 0.8 2024-04-25T23:48:03.273 INFO:teuthology.orchestra.run.smithi080.stdout: osd debug verify cached snaps = True 2024-04-25T23:48:03.273 INFO:teuthology.orchestra.run.smithi080.stdout: osd debug verify missing on start = True 2024-04-25T23:48:03.273 INFO:teuthology.orchestra.run.smithi080.stdout: osd failsafe full ratio = 0.95 2024-04-25T23:48:03.274 INFO:teuthology.orchestra.run.smithi080.stdout: osd mclock override recovery settings = True 2024-04-25T23:48:03.274 INFO:teuthology.orchestra.run.smithi080.stdout: osd objectstore = bluestore 2024-04-25T23:48:03.274 INFO:teuthology.orchestra.run.smithi080.stdout: osd op queue = debug_random 2024-04-25T23:48:03.274 INFO:teuthology.orchestra.run.smithi080.stdout: osd op queue cut off = debug_random 2024-04-25T23:48:03.274 INFO:teuthology.orchestra.run.smithi080.stdout: osd shutdown pgref assert = True 2024-04-25T23:48:03.274 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T23:48:03.274 INFO:teuthology.orchestra.run.smithi080.stdout:[mgr] 2024-04-25T23:48:03.274 INFO:teuthology.orchestra.run.smithi080.stdout: debug ms = 1 2024-04-25T23:48:03.274 INFO:teuthology.orchestra.run.smithi080.stdout: debug mgr = 20 2024-04-25T23:48:03.274 INFO:teuthology.orchestra.run.smithi080.stdout: debug mon = 20 2024-04-25T23:48:03.274 INFO:teuthology.orchestra.run.smithi080.stdout: debug auth = 20 2024-04-25T23:48:03.274 INFO:teuthology.orchestra.run.smithi080.stdout: mon reweight min pgs per osd = 4 2024-04-25T23:48:03.274 INFO:teuthology.orchestra.run.smithi080.stdout: mon reweight min bytes per osd = 10 2024-04-25T23:48:03.274 INFO:teuthology.orchestra.run.smithi080.stdout: mgr/telemetry/nag = false 2024-04-25T23:48:03.274 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T23:48:03.274 INFO:teuthology.orchestra.run.smithi080.stdout:[mon] 2024-04-25T23:48:03.275 INFO:teuthology.orchestra.run.smithi080.stdout: debug ms = 1 2024-04-25T23:48:03.275 INFO:teuthology.orchestra.run.smithi080.stdout: debug mon = 20 2024-04-25T23:48:03.275 INFO:teuthology.orchestra.run.smithi080.stdout: debug paxos = 20 2024-04-25T23:48:03.275 INFO:teuthology.orchestra.run.smithi080.stdout: debug auth = 20 2024-04-25T23:48:03.275 INFO:teuthology.orchestra.run.smithi080.stdout: mon data avail warn = 5 2024-04-25T23:48:03.275 INFO:teuthology.orchestra.run.smithi080.stdout: mon mgr mkfs grace = 240 2024-04-25T23:48:03.275 INFO:teuthology.orchestra.run.smithi080.stdout: mon reweight min pgs per osd = 4 2024-04-25T23:48:03.275 INFO:teuthology.orchestra.run.smithi080.stdout: mon osd reporter subtree level = osd 2024-04-25T23:48:03.275 INFO:teuthology.orchestra.run.smithi080.stdout: mon osd prime pg temp = true 2024-04-25T23:48:03.275 INFO:teuthology.orchestra.run.smithi080.stdout: mon reweight min bytes per osd = 10 2024-04-25T23:48:03.275 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T23:48:03.275 INFO:teuthology.orchestra.run.smithi080.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T23:48:03.275 INFO:teuthology.orchestra.run.smithi080.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T23:48:03.275 INFO:teuthology.orchestra.run.smithi080.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T23:48:03.275 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T23:48:03.275 INFO:teuthology.orchestra.run.smithi080.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T23:48:03.276 INFO:teuthology.orchestra.run.smithi080.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T23:48:03.276 INFO:teuthology.orchestra.run.smithi080.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T23:48:03.276 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T23:48:03.276 INFO:teuthology.orchestra.run.smithi080.stdout: # 1m isn't quite enough 2024-04-25T23:48:03.276 INFO:teuthology.orchestra.run.smithi080.stdout: mon_down_mkfs_grace = 2m 2024-04-25T23:48:03.276 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T23:48:03.276 INFO:teuthology.orchestra.run.smithi080.stdout: mon_warn_on_filestore_osds = false 2024-04-25T23:48:03.276 INFO:teuthology.orchestra.run.smithi080.stdout: mon min osdmap epochs = 50 2024-04-25T23:48:03.276 INFO:teuthology.orchestra.run.smithi080.stdout: mon osdmap full prune interval = 2 2024-04-25T23:48:03.276 INFO:teuthology.orchestra.run.smithi080.stdout: mon osdmap full prune min = 15 2024-04-25T23:48:03.276 INFO:teuthology.orchestra.run.smithi080.stdout: mon osdmap full prune txsize = 2 2024-04-25T23:48:03.276 INFO:teuthology.orchestra.run.smithi080.stdout: mon scrub interval = 300 2024-04-25T23:48:03.276 INFO:teuthology.orchestra.run.smithi080.stdout: paxos service trim min = 10 2024-04-25T23:48:03.276 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T23:48:03.276 INFO:teuthology.orchestra.run.smithi080.stdout:[client] 2024-04-25T23:48:03.276 INFO:teuthology.orchestra.run.smithi080.stdout: rgw cache enabled = true 2024-04-25T23:48:03.277 INFO:teuthology.orchestra.run.smithi080.stdout: rgw enable ops log = true 2024-04-25T23:48:03.277 INFO:teuthology.orchestra.run.smithi080.stdout: rgw enable usage log = true 2024-04-25T23:48:03.277 INFO:teuthology.orchestra.run.smithi080.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T23:48:03.277 INFO:teuthology.orchestra.run.smithi080.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T23:48:03.277 INFO:teuthology.orchestra.run.smithi080.stdout: debug cephsqlite = 20 2024-04-25T23:48:03.277 INFO:teuthology.orchestra.run.smithi080.stdout: debug client = 20 2024-04-25T23:48:03.277 INFO:teuthology.orchestra.run.smithi080.stdout: debug ms = 1 2024-04-25T23:48:03.277 INFO:teuthology.orchestra.run.smithi080.stdout:[mon.a] 2024-04-25T23:48:03.277 INFO:teuthology.orchestra.run.smithi080.stdout:[mon.c] 2024-04-25T23:48:03.277 INFO:teuthology.orchestra.run.smithi080.stdout:[mon.b] 2024-04-25T23:48:03.277 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-25T23:48:03.277 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-25T23:48:03.360 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-25T23:48:03.360 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-04-25T23:48:03.360 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-25T23:48:03.365 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-04-25T23:48:03.365 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-25T23:48:03.414 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi057.front.sepia.ceph.com 2024-04-25T23:48:03.414 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-04-25T23:48:03.414 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T23:48:03.414 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T23:48:03.480 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-04-25T23:48:03.481 DEBUG:teuthology.orchestra.run.smithi057:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T23:48:03.527 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi080.front.sepia.ceph.com 2024-04-25T23:48:03.527 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T23:48:03.527 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T23:48:03.527 DEBUG:teuthology.orchestra.run.smithi080:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T23:48:03.549 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T23:48:03.550 DEBUG:teuthology.orchestra.run.smithi080:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T23:48:03.599 INFO:tasks.ceph:Setting up mon nodes... 2024-04-25T23:48:03.599 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-25T23:48:03.599 DEBUG:teuthology.orchestra.run.smithi057:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T23:48:03.645 INFO:teuthology.orchestra.run.smithi057.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T23:48:03.647 DEBUG:teuthology.orchestra.run.smithi080:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T23:48:03.690 INFO:teuthology.orchestra.run.smithi080.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T23:48:03.701 INFO:tasks.ceph:Setting up mds nodes... 2024-04-25T23:48:03.701 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-25T23:48:03.702 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T23:48:03.732 INFO:teuthology.orchestra.run.smithi057.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-25T23:48:03.741 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-25T23:48:03.785 INFO:teuthology.orchestra.run.smithi080.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-25T23:48:03.798 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-25T23:48:03.798 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi057.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi080.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T23:48:03.798 DEBUG:teuthology.orchestra.run.smithi057:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-25T23:48:03.810 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T23:48:03.810 INFO:tasks.ceph:role: osd.0 2024-04-25T23:48:03.810 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi057.front.sepia.ceph.com 2024-04-25T23:48:03.810 DEBUG:teuthology.orchestra.run.smithi057:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T23:48:03.897 INFO:teuthology.orchestra.run.smithi057.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T23:48:03.897 INFO:teuthology.orchestra.run.smithi057.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T23:48:03.897 INFO:teuthology.orchestra.run.smithi057.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T23:48:03.897 INFO:teuthology.orchestra.run.smithi057.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T23:48:03.897 INFO:teuthology.orchestra.run.smithi057.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T23:48:03.897 INFO:teuthology.orchestra.run.smithi057.stdout: = sunit=0 swidth=0 blks 2024-04-25T23:48:03.897 INFO:teuthology.orchestra.run.smithi057.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T23:48:03.897 INFO:teuthology.orchestra.run.smithi057.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T23:48:03.898 INFO:teuthology.orchestra.run.smithi057.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T23:48:03.898 INFO:teuthology.orchestra.run.smithi057.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T23:48:05.181 INFO:teuthology.orchestra.run.smithi057.stdout:Discarding blocks...Done. 2024-04-25T23:48:05.182 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi057.front.sepia.ceph.com -o noatime 2024-04-25T23:48:05.182 DEBUG:teuthology.orchestra.run.smithi057:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-25T23:48:05.248 DEBUG:teuthology.orchestra.run.smithi057:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-25T23:48:05.303 INFO:teuthology.orchestra.run.smithi057.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T23:48:05.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T23:48:05.304 DEBUG:teuthology.orchestra.run.smithi057:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-25T23:48:05.358 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T23:48:05.359 INFO:tasks.ceph:role: osd.1 2024-04-25T23:48:05.359 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi057.front.sepia.ceph.com 2024-04-25T23:48:05.359 DEBUG:teuthology.orchestra.run.smithi057:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T23:48:05.442 INFO:teuthology.orchestra.run.smithi057.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T23:48:05.443 INFO:teuthology.orchestra.run.smithi057.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T23:48:05.443 INFO:teuthology.orchestra.run.smithi057.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T23:48:05.443 INFO:teuthology.orchestra.run.smithi057.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T23:48:05.443 INFO:teuthology.orchestra.run.smithi057.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T23:48:05.443 INFO:teuthology.orchestra.run.smithi057.stdout: = sunit=0 swidth=0 blks 2024-04-25T23:48:05.443 INFO:teuthology.orchestra.run.smithi057.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T23:48:05.443 INFO:teuthology.orchestra.run.smithi057.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T23:48:05.443 INFO:teuthology.orchestra.run.smithi057.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T23:48:05.443 INFO:teuthology.orchestra.run.smithi057.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T23:48:06.776 INFO:teuthology.orchestra.run.smithi057.stdout:Discarding blocks...Done. 2024-04-25T23:48:06.778 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi057.front.sepia.ceph.com -o noatime 2024-04-25T23:48:06.779 DEBUG:teuthology.orchestra.run.smithi057:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-25T23:48:06.845 DEBUG:teuthology.orchestra.run.smithi057:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-25T23:48:06.900 INFO:teuthology.orchestra.run.smithi057.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T23:48:06.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T23:48:06.901 DEBUG:teuthology.orchestra.run.smithi057:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-25T23:48:06.958 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T23:48:06.958 INFO:tasks.ceph:role: osd.2 2024-04-25T23:48:06.958 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi057.front.sepia.ceph.com 2024-04-25T23:48:06.958 DEBUG:teuthology.orchestra.run.smithi057:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T23:48:07.044 INFO:teuthology.orchestra.run.smithi057.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T23:48:07.044 INFO:teuthology.orchestra.run.smithi057.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T23:48:07.044 INFO:teuthology.orchestra.run.smithi057.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T23:48:07.044 INFO:teuthology.orchestra.run.smithi057.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T23:48:07.044 INFO:teuthology.orchestra.run.smithi057.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T23:48:07.045 INFO:teuthology.orchestra.run.smithi057.stdout: = sunit=0 swidth=0 blks 2024-04-25T23:48:07.045 INFO:teuthology.orchestra.run.smithi057.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T23:48:07.045 INFO:teuthology.orchestra.run.smithi057.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T23:48:07.045 INFO:teuthology.orchestra.run.smithi057.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T23:48:07.045 INFO:teuthology.orchestra.run.smithi057.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T23:48:08.349 INFO:teuthology.orchestra.run.smithi057.stdout:Discarding blocks...Done. 2024-04-25T23:48:08.351 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi057.front.sepia.ceph.com -o noatime 2024-04-25T23:48:08.352 DEBUG:teuthology.orchestra.run.smithi057:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-25T23:48:08.418 DEBUG:teuthology.orchestra.run.smithi057:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-25T23:48:08.471 INFO:teuthology.orchestra.run.smithi057.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T23:48:08.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T23:48:08.472 DEBUG:teuthology.orchestra.run.smithi057:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-25T23:48:08.526 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T23:48:08.526 INFO:tasks.ceph:role: osd.3 2024-04-25T23:48:08.526 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi057.front.sepia.ceph.com 2024-04-25T23:48:08.526 DEBUG:teuthology.orchestra.run.smithi057:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T23:48:08.610 INFO:teuthology.orchestra.run.smithi057.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T23:48:08.611 INFO:teuthology.orchestra.run.smithi057.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T23:48:08.611 INFO:teuthology.orchestra.run.smithi057.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T23:48:08.611 INFO:teuthology.orchestra.run.smithi057.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T23:48:08.611 INFO:teuthology.orchestra.run.smithi057.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T23:48:08.611 INFO:teuthology.orchestra.run.smithi057.stdout: = sunit=0 swidth=0 blks 2024-04-25T23:48:08.611 INFO:teuthology.orchestra.run.smithi057.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T23:48:08.611 INFO:teuthology.orchestra.run.smithi057.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T23:48:08.611 INFO:teuthology.orchestra.run.smithi057.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T23:48:08.611 INFO:teuthology.orchestra.run.smithi057.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T23:48:09.914 INFO:teuthology.orchestra.run.smithi057.stdout:Discarding blocks...Done. 2024-04-25T23:48:09.916 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi057.front.sepia.ceph.com -o noatime 2024-04-25T23:48:09.916 DEBUG:teuthology.orchestra.run.smithi057:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-25T23:48:09.981 DEBUG:teuthology.orchestra.run.smithi057:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-25T23:48:10.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T23:48:10.030 INFO:teuthology.orchestra.run.smithi057.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T23:48:10.031 DEBUG:teuthology.orchestra.run.smithi057:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T23:48:10.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:10.148+0000 7f04827447c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-25T23:48:10.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:10.148+0000 7f04827447c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-25T23:48:10.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:10.148+0000 7f04827447c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T23:48:10.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:10.148+0000 7f04827447c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T23:48:10.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:10.148+0000 7f04827447c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T23:48:10.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:10.152+0000 7f04827447c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-25T23:48:13.137 DEBUG:teuthology.orchestra.run.smithi057:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-25T23:48:13.151 DEBUG:teuthology.orchestra.run.smithi057:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T23:48:13.237 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:13.248+0000 7fad6826f7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-25T23:48:13.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:13.252+0000 7fad6826f7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-25T23:48:13.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:13.252+0000 7fad6826f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T23:48:13.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:13.252+0000 7fad6826f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T23:48:13.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:13.252+0000 7fad6826f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T23:48:13.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:13.252+0000 7fad6826f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-25T23:48:16.234 DEBUG:teuthology.orchestra.run.smithi057:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-25T23:48:16.291 DEBUG:teuthology.orchestra.run.smithi057:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T23:48:16.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:16.392+0000 7ff3ca5d87c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-25T23:48:16.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:16.392+0000 7ff3ca5d87c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-25T23:48:16.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:16.392+0000 7ff3ca5d87c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T23:48:16.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:16.392+0000 7ff3ca5d87c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T23:48:16.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:16.392+0000 7ff3ca5d87c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T23:48:16.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:16.392+0000 7ff3ca5d87c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-25T23:48:19.362 DEBUG:teuthology.orchestra.run.smithi057:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-25T23:48:19.419 DEBUG:teuthology.orchestra.run.smithi057:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T23:48:19.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:19.512+0000 7f9f675ef7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-25T23:48:19.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:19.516+0000 7f9f675ef7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-25T23:48:19.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:19.516+0000 7f9f675ef7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T23:48:19.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:19.516+0000 7f9f675ef7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T23:48:19.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:19.516+0000 7f9f675ef7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T23:48:19.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:19.516+0000 7f9f675ef7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-25T23:48:22.502 DEBUG:teuthology.orchestra.run.smithi057:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-25T23:48:22.559 DEBUG:teuthology.orchestra.run.smithi080:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-25T23:48:22.574 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T23:48:22.574 INFO:tasks.ceph:role: osd.4 2024-04-25T23:48:22.574 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi080.front.sepia.ceph.com 2024-04-25T23:48:22.574 DEBUG:teuthology.orchestra.run.smithi080:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T23:48:22.637 INFO:teuthology.orchestra.run.smithi080.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T23:48:22.637 INFO:teuthology.orchestra.run.smithi080.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T23:48:22.637 INFO:teuthology.orchestra.run.smithi080.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T23:48:22.637 INFO:teuthology.orchestra.run.smithi080.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T23:48:22.637 INFO:teuthology.orchestra.run.smithi080.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T23:48:22.637 INFO:teuthology.orchestra.run.smithi080.stdout: = sunit=0 swidth=0 blks 2024-04-25T23:48:22.637 INFO:teuthology.orchestra.run.smithi080.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T23:48:22.637 INFO:teuthology.orchestra.run.smithi080.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T23:48:22.638 INFO:teuthology.orchestra.run.smithi080.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T23:48:22.638 INFO:teuthology.orchestra.run.smithi080.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T23:48:23.080 INFO:teuthology.orchestra.run.smithi080.stdout:Discarding blocks...Done. 2024-04-25T23:48:23.082 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi080.front.sepia.ceph.com -o noatime 2024-04-25T23:48:23.082 DEBUG:teuthology.orchestra.run.smithi080:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-25T23:48:23.173 DEBUG:teuthology.orchestra.run.smithi080:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-25T23:48:23.183 INFO:teuthology.orchestra.run.smithi080.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T23:48:23.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T23:48:23.184 DEBUG:teuthology.orchestra.run.smithi080:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-25T23:48:23.237 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T23:48:23.237 INFO:tasks.ceph:role: osd.5 2024-04-25T23:48:23.237 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi080.front.sepia.ceph.com 2024-04-25T23:48:23.237 DEBUG:teuthology.orchestra.run.smithi080:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T23:48:23.300 INFO:teuthology.orchestra.run.smithi080.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T23:48:23.301 INFO:teuthology.orchestra.run.smithi080.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T23:48:23.301 INFO:teuthology.orchestra.run.smithi080.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T23:48:23.301 INFO:teuthology.orchestra.run.smithi080.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T23:48:23.301 INFO:teuthology.orchestra.run.smithi080.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T23:48:23.301 INFO:teuthology.orchestra.run.smithi080.stdout: = sunit=0 swidth=0 blks 2024-04-25T23:48:23.301 INFO:teuthology.orchestra.run.smithi080.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T23:48:23.301 INFO:teuthology.orchestra.run.smithi080.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T23:48:23.301 INFO:teuthology.orchestra.run.smithi080.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T23:48:23.301 INFO:teuthology.orchestra.run.smithi080.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T23:48:23.768 INFO:teuthology.orchestra.run.smithi080.stdout:Discarding blocks...Done. 2024-04-25T23:48:23.769 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi080.front.sepia.ceph.com -o noatime 2024-04-25T23:48:23.770 DEBUG:teuthology.orchestra.run.smithi080:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-25T23:48:23.875 DEBUG:teuthology.orchestra.run.smithi080:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-25T23:48:23.885 INFO:teuthology.orchestra.run.smithi080.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T23:48:23.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T23:48:23.886 DEBUG:teuthology.orchestra.run.smithi080:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-25T23:48:23.941 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T23:48:23.941 INFO:tasks.ceph:role: osd.6 2024-04-25T23:48:23.941 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi080.front.sepia.ceph.com 2024-04-25T23:48:23.941 DEBUG:teuthology.orchestra.run.smithi080:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T23:48:24.004 INFO:teuthology.orchestra.run.smithi080.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T23:48:24.004 INFO:teuthology.orchestra.run.smithi080.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T23:48:24.004 INFO:teuthology.orchestra.run.smithi080.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T23:48:24.005 INFO:teuthology.orchestra.run.smithi080.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T23:48:24.005 INFO:teuthology.orchestra.run.smithi080.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T23:48:24.005 INFO:teuthology.orchestra.run.smithi080.stdout: = sunit=0 swidth=0 blks 2024-04-25T23:48:24.005 INFO:teuthology.orchestra.run.smithi080.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T23:48:24.005 INFO:teuthology.orchestra.run.smithi080.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T23:48:24.006 INFO:teuthology.orchestra.run.smithi080.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T23:48:24.006 INFO:teuthology.orchestra.run.smithi080.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T23:48:24.500 INFO:teuthology.orchestra.run.smithi080.stdout:Discarding blocks...Done. 2024-04-25T23:48:24.502 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi080.front.sepia.ceph.com -o noatime 2024-04-25T23:48:24.502 DEBUG:teuthology.orchestra.run.smithi080:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-25T23:48:24.649 DEBUG:teuthology.orchestra.run.smithi080:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-25T23:48:24.660 INFO:teuthology.orchestra.run.smithi080.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T23:48:24.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T23:48:24.661 DEBUG:teuthology.orchestra.run.smithi080:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-25T23:48:24.718 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T23:48:24.718 INFO:tasks.ceph:role: osd.7 2024-04-25T23:48:24.718 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi080.front.sepia.ceph.com 2024-04-25T23:48:24.718 DEBUG:teuthology.orchestra.run.smithi080:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T23:48:24.786 INFO:teuthology.orchestra.run.smithi080.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T23:48:24.786 INFO:teuthology.orchestra.run.smithi080.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T23:48:24.786 INFO:teuthology.orchestra.run.smithi080.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T23:48:24.786 INFO:teuthology.orchestra.run.smithi080.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T23:48:24.786 INFO:teuthology.orchestra.run.smithi080.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T23:48:24.786 INFO:teuthology.orchestra.run.smithi080.stdout: = sunit=0 swidth=0 blks 2024-04-25T23:48:24.786 INFO:teuthology.orchestra.run.smithi080.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T23:48:24.786 INFO:teuthology.orchestra.run.smithi080.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T23:48:24.786 INFO:teuthology.orchestra.run.smithi080.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T23:48:24.786 INFO:teuthology.orchestra.run.smithi080.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T23:48:25.198 INFO:teuthology.orchestra.run.smithi080.stdout:Discarding blocks...Done. 2024-04-25T23:48:25.200 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi080.front.sepia.ceph.com -o noatime 2024-04-25T23:48:25.200 DEBUG:teuthology.orchestra.run.smithi080:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-25T23:48:25.295 DEBUG:teuthology.orchestra.run.smithi080:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-25T23:48:25.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T23:48:25.304 INFO:teuthology.orchestra.run.smithi080.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T23:48:25.304 DEBUG:teuthology.orchestra.run.smithi080:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T23:48:25.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:25.421+0000 7ff45285b7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-25T23:48:25.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:25.421+0000 7ff45285b7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-25T23:48:25.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:25.421+0000 7ff45285b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T23:48:25.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:25.421+0000 7ff45285b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T23:48:25.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:25.421+0000 7ff45285b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T23:48:25.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:25.429+0000 7ff45285b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-25T23:48:28.428 DEBUG:teuthology.orchestra.run.smithi080:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-25T23:48:28.443 DEBUG:teuthology.orchestra.run.smithi080:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T23:48:28.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:28.537+0000 7f54a3f1a7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-25T23:48:28.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:28.537+0000 7f54a3f1a7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-25T23:48:28.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:28.537+0000 7f54a3f1a7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T23:48:28.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:28.537+0000 7f54a3f1a7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T23:48:28.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:28.537+0000 7f54a3f1a7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T23:48:28.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:28.541+0000 7f54a3f1a7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-25T23:48:31.493 DEBUG:teuthology.orchestra.run.smithi080:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-25T23:48:31.549 DEBUG:teuthology.orchestra.run.smithi080:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T23:48:31.631 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:31.645+0000 7f48b38057c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-25T23:48:31.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:31.645+0000 7f48b38057c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-25T23:48:31.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:31.645+0000 7f48b38057c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T23:48:31.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:31.645+0000 7f48b38057c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T23:48:31.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:31.645+0000 7f48b38057c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T23:48:31.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:31.645+0000 7f48b38057c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-25T23:48:34.597 DEBUG:teuthology.orchestra.run.smithi080:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-25T23:48:34.654 DEBUG:teuthology.orchestra.run.smithi080:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T23:48:34.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:34.749+0000 7fc663de77c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-25T23:48:34.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:34.749+0000 7fc663de77c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-25T23:48:34.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:34.749+0000 7fc663de77c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T23:48:34.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:34.749+0000 7fc663de77c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T23:48:34.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:34.749+0000 7fc663de77c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T23:48:34.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:34.749+0000 7fc663de77c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-25T23:48:37.705 DEBUG:teuthology.orchestra.run.smithi080:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-25T23:48:37.761 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-25T23:48:37.761 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-04-25T23:48:37.762 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-25T23:48:37.776 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-04-25T23:48:37.776 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-25T23:48:37.830 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-04-25T23:48:37.830 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-25T23:48:37.885 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-04-25T23:48:37.885 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-25T23:48:37.942 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-04-25T23:48:37.942 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-25T23:48:37.997 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T23:48:37.998 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-25T23:48:38.011 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T23:48:38.011 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-25T23:48:38.064 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T23:48:38.064 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-25T23:48:38.116 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T23:48:38.116 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-25T23:48:38.173 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T23:48:38.173 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-25T23:48:38.229 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-04-25T23:48:38.229 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-25T23:48:38.235 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T23:48:38.235 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-25T23:48:38.277 INFO:tasks.ceph:Adding keys to all mons... 2024-04-25T23:48:38.277 DEBUG:teuthology.orchestra.run.smithi057:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T23:48:38.280 DEBUG:teuthology.orchestra.run.smithi080:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T23:48:38.324 INFO:teuthology.orchestra.run.smithi057.stdout:[mgr.y] 2024-04-25T23:48:38.324 INFO:teuthology.orchestra.run.smithi057.stdout: key = AQCz6ypm7OFtJxAAyNA1RQNgfjxGAtyTnT1hOw== 2024-04-25T23:48:38.324 INFO:teuthology.orchestra.run.smithi057.stdout:[osd.0] 2024-04-25T23:48:38.324 INFO:teuthology.orchestra.run.smithi057.stdout: key = AQC66ypmSu8aCRAAFZD5dZSO5xtZbcPLA8PT7w== 2024-04-25T23:48:38.324 INFO:teuthology.orchestra.run.smithi057.stdout:[osd.1] 2024-04-25T23:48:38.324 INFO:teuthology.orchestra.run.smithi057.stdout: key = AQC96ypmb8UrDxAA7goC+KnoLLCRSlzbCdi2+A== 2024-04-25T23:48:38.324 INFO:teuthology.orchestra.run.smithi057.stdout:[osd.2] 2024-04-25T23:48:38.324 INFO:teuthology.orchestra.run.smithi057.stdout: key = AQDA6ypmwqeUFxAA2P58dJoN2nfa5WHGfLgvOg== 2024-04-25T23:48:38.324 INFO:teuthology.orchestra.run.smithi057.stdout:[osd.3] 2024-04-25T23:48:38.324 INFO:teuthology.orchestra.run.smithi057.stdout: key = AQDD6ypmw+/iHhAA1yXo37Y1I95/AOxkALwFrw== 2024-04-25T23:48:38.325 INFO:teuthology.orchestra.run.smithi057.stdout:[mgr.x] 2024-04-25T23:48:38.325 INFO:teuthology.orchestra.run.smithi057.stdout: key = AQCz6ypmZU1IKhAA/rcTFp+V41UWLcuLlF8A9w== 2024-04-25T23:48:38.325 INFO:teuthology.orchestra.run.smithi057.stdout:[osd.4] 2024-04-25T23:48:38.325 INFO:teuthology.orchestra.run.smithi057.stdout: key = AQDJ6ypmE45zGRAAGmXuV8qpAQDunpQXTrdqBA== 2024-04-25T23:48:38.325 INFO:teuthology.orchestra.run.smithi057.stdout:[osd.5] 2024-04-25T23:48:38.325 INFO:teuthology.orchestra.run.smithi057.stdout: key = AQDM6ypmpw5rIBAAhzJMw7B/6Btvlt7yPwv99Q== 2024-04-25T23:48:38.325 INFO:teuthology.orchestra.run.smithi057.stdout:[osd.6] 2024-04-25T23:48:38.325 INFO:teuthology.orchestra.run.smithi057.stdout: key = AQDP6ypm5CjPJhAAZormUc/knOTpgO4TqzevJw== 2024-04-25T23:48:38.325 INFO:teuthology.orchestra.run.smithi057.stdout:[osd.7] 2024-04-25T23:48:38.325 INFO:teuthology.orchestra.run.smithi057.stdout: key = AQDS6ypmW04DLRAAXb5LgmoPTG6/x9/5P6d8Aw== 2024-04-25T23:48:38.325 INFO:teuthology.orchestra.run.smithi057.stdout:[client.0] 2024-04-25T23:48:38.325 INFO:teuthology.orchestra.run.smithi057.stdout: key = AQCz6ypmQdqhLBAA3JA47fda/es5Sp+2xZjuJw== 2024-04-25T23:48:38.326 INFO:teuthology.orchestra.run.smithi057.stdout:[client.1] 2024-04-25T23:48:38.326 INFO:teuthology.orchestra.run.smithi057.stdout: key = AQCz6ypm+47uLxAAZ42/OTwZO8TBJhPnTA0oWg== 2024-04-25T23:48:38.331 INFO:teuthology.orchestra.run.smithi080.stdout:[mgr.y] 2024-04-25T23:48:38.331 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQCz6ypm7OFtJxAAyNA1RQNgfjxGAtyTnT1hOw== 2024-04-25T23:48:38.332 INFO:teuthology.orchestra.run.smithi080.stdout:[osd.0] 2024-04-25T23:48:38.332 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQC66ypmSu8aCRAAFZD5dZSO5xtZbcPLA8PT7w== 2024-04-25T23:48:38.332 INFO:teuthology.orchestra.run.smithi080.stdout:[osd.1] 2024-04-25T23:48:38.332 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQC96ypmb8UrDxAA7goC+KnoLLCRSlzbCdi2+A== 2024-04-25T23:48:38.332 INFO:teuthology.orchestra.run.smithi080.stdout:[osd.2] 2024-04-25T23:48:38.332 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQDA6ypmwqeUFxAA2P58dJoN2nfa5WHGfLgvOg== 2024-04-25T23:48:38.332 INFO:teuthology.orchestra.run.smithi080.stdout:[osd.3] 2024-04-25T23:48:38.332 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQDD6ypmw+/iHhAA1yXo37Y1I95/AOxkALwFrw== 2024-04-25T23:48:38.332 INFO:teuthology.orchestra.run.smithi080.stdout:[mgr.x] 2024-04-25T23:48:38.332 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQCz6ypmZU1IKhAA/rcTFp+V41UWLcuLlF8A9w== 2024-04-25T23:48:38.332 INFO:teuthology.orchestra.run.smithi080.stdout:[osd.4] 2024-04-25T23:48:38.332 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQDJ6ypmE45zGRAAGmXuV8qpAQDunpQXTrdqBA== 2024-04-25T23:48:38.333 INFO:teuthology.orchestra.run.smithi080.stdout:[osd.5] 2024-04-25T23:48:38.333 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQDM6ypmpw5rIBAAhzJMw7B/6Btvlt7yPwv99Q== 2024-04-25T23:48:38.333 INFO:teuthology.orchestra.run.smithi080.stdout:[osd.6] 2024-04-25T23:48:38.333 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQDP6ypm5CjPJhAAZormUc/knOTpgO4TqzevJw== 2024-04-25T23:48:38.333 INFO:teuthology.orchestra.run.smithi080.stdout:[osd.7] 2024-04-25T23:48:38.333 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQDS6ypmW04DLRAAXb5LgmoPTG6/x9/5P6d8Aw== 2024-04-25T23:48:38.333 INFO:teuthology.orchestra.run.smithi080.stdout:[client.0] 2024-04-25T23:48:38.333 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQCz6ypmQdqhLBAA3JA47fda/es5Sp+2xZjuJw== 2024-04-25T23:48:38.333 INFO:teuthology.orchestra.run.smithi080.stdout:[client.1] 2024-04-25T23:48:38.333 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQCz6ypm+47uLxAAZ42/OTwZO8TBJhPnTA0oWg== 2024-04-25T23:48:38.334 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T23:48:38.371 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T23:48:38.422 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T23:48:38.427 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T23:48:38.467 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T23:48:38.511 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T23:48:38.553 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T23:48:38.595 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T23:48:38.639 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T23:48:38.679 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T23:48:38.725 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T23:48:38.763 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T23:48:38.812 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T23:48:38.847 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T23:48:38.892 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T23:48:38.935 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T23:48:38.975 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T23:48:39.019 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T23:48:39.064 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T23:48:39.107 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T23:48:39.147 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T23:48:39.190 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T23:48:39.234 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T23:48:39.236 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T23:48:39.323 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-25T23:48:39.323 DEBUG:teuthology.orchestra.run.smithi057:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-25T23:48:39.336 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T23:48:39.433 DEBUG:teuthology.orchestra.run.smithi057:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-25T23:48:39.446 DEBUG:teuthology.orchestra.run.smithi057:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-25T23:48:39.499 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T23:48:39.598 DEBUG:teuthology.orchestra.run.smithi057:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-25T23:48:39.609 DEBUG:teuthology.orchestra.run.smithi080:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-25T23:48:39.623 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T23:48:39.724 DEBUG:teuthology.orchestra.run.smithi080:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-25T23:48:39.734 DEBUG:teuthology.orchestra.run.smithi057:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T23:48:39.736 DEBUG:teuthology.orchestra.run.smithi080:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T23:48:39.781 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-25T23:48:39.781 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-25T23:48:39.781 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-25T23:48:39.784 INFO:tasks.ceph.mon.a:Started 2024-04-25T23:48:39.784 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-25T23:48:39.784 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-25T23:48:39.786 INFO:tasks.ceph.mon.c:Started 2024-04-25T23:48:39.786 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-25T23:48:39.786 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-25T23:48:39.826 INFO:tasks.ceph.mon.b:Started 2024-04-25T23:48:39.826 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-25T23:48:39.827 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-25T23:48:39.827 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-25T23:48:39.829 INFO:tasks.ceph.mgr.y:Started 2024-04-25T23:48:39.829 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-25T23:48:39.829 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-25T23:48:39.832 INFO:tasks.ceph.mgr.x:Started 2024-04-25T23:48:39.832 INFO:tasks.ceph:Setting crush tunables to default 2024-04-25T23:48:39.832 DEBUG:teuthology.orchestra.run.smithi057:> sudo ceph --cluster ceph osd crush tunables default 2024-04-25T23:48:39.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:39.963+0000 7f4b65d2d640 1 Processor -- start 2024-04-25T23:48:39.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:39.963+0000 7f4b65d2d640 1 -- start start 2024-04-25T23:48:39.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:39.963+0000 7f4b65d2d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f4b6013c850 0x7f4b6014ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:39.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:39.963+0000 7f4b65d2d640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b60072610 con 0x7f4b60073050 2024-04-25T23:48:39.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:39.963+0000 7f4b65d2d640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b600728f0 con 0x7f4b6013f950 2024-04-25T23:48:39.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:39.963+0000 7f4b65d2d640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f4b60071780 con 0x7f4b6013c850 2024-04-25T23:48:39.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:39.963+0000 7f4b5f7fe640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f4b60073050 0x7f4b6013f240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:35758/0 (socket says 172.21.15.57:35758) 2024-04-25T23:48:39.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:39.963+0000 7f4b5f7fe640 1 -- 172.21.15.57:0/2640779937 learned_addr learned my addr 172.21.15.57:0/2640779937 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-04-25T23:48:39.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:39.963+0000 7f4b5ffff640 1 --2- 172.21.15.57:0/2640779937 >> v2:172.21.15.80:3300/0 conn(0x7f4b6013c850 0x7f4b6014ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:39.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:39.963+0000 7f4b5ffff640 1 -- 172.21.15.57:0/2640779937 >> v2:172.21.15.80:3300/0 conn(0x7f4b6013c850 msgr2=0x7f4b6014ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T23:48:39.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:39.963+0000 7f4b5ffff640 1 -- 172.21.15.57:0/2640779937 >> v2:172.21.15.80:3300/0 conn(0x7f4b6013c850 msgr2=0x7f4b6014ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T23:48:39.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:39.963+0000 7f4b5ffff640 1 --2- 172.21.15.57:0/2640779937 >> v2:172.21.15.80:3300/0 conn(0x7f4b6013c850 0x7f4b6014ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T23:48:39.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:39.963+0000 7f4b5ffff640 1 --2- 172.21.15.57:0/2640779937 >> v2:172.21.15.80:3300/0 conn(0x7f4b6013c850 0x7f4b6014ab60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T23:48:40.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:40.167+0000 7f4b5ffff640 1 --2- 172.21.15.57:0/2640779937 >> v2:172.21.15.80:3300/0 conn(0x7f4b6013c850 0x7f4b6014ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:40.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:40.167+0000 7f4b5ffff640 1 -- 172.21.15.57:0/2640779937 >> v2:172.21.15.80:3300/0 conn(0x7f4b6013c850 msgr2=0x7f4b6014ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T23:48:40.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:40.167+0000 7f4b5ffff640 1 -- 172.21.15.57:0/2640779937 >> v2:172.21.15.80:3300/0 conn(0x7f4b6013c850 msgr2=0x7f4b6014ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T23:48:40.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:40.167+0000 7f4b5ffff640 1 --2- 172.21.15.57:0/2640779937 >> v2:172.21.15.80:3300/0 conn(0x7f4b6013c850 0x7f4b6014ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T23:48:40.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:40.167+0000 7f4b5ffff640 1 --2- 172.21.15.57:0/2640779937 >> v2:172.21.15.80:3300/0 conn(0x7f4b6013c850 0x7f4b6014ab60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T23:48:40.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:40.571+0000 7f4b5ffff640 1 --2- 172.21.15.57:0/2640779937 >> v2:172.21.15.80:3300/0 conn(0x7f4b6013c850 0x7f4b6014ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:40.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:40.571+0000 7f4b5ffff640 1 -- 172.21.15.57:0/2640779937 >> v2:172.21.15.80:3300/0 conn(0x7f4b6013c850 msgr2=0x7f4b6014ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T23:48:40.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:40.571+0000 7f4b5ffff640 1 -- 172.21.15.57:0/2640779937 >> v2:172.21.15.80:3300/0 conn(0x7f4b6013c850 msgr2=0x7f4b6014ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T23:48:40.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:40.571+0000 7f4b5ffff640 1 --2- 172.21.15.57:0/2640779937 >> v2:172.21.15.80:3300/0 conn(0x7f4b6013c850 0x7f4b6014ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T23:48:40.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:40.571+0000 7f4b5ffff640 1 --2- 172.21.15.57:0/2640779937 >> v2:172.21.15.80:3300/0 conn(0x7f4b6013c850 0x7f4b6014ab60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T23:48:41.362 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:41.375+0000 7f4b5ffff640 1 --2- 172.21.15.57:0/2640779937 >> v2:172.21.15.80:3300/0 conn(0x7f4b6013c850 0x7f4b6014ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:41.362 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:41.375+0000 7f4b5ffff640 1 -- 172.21.15.57:0/2640779937 >> v2:172.21.15.80:3300/0 conn(0x7f4b6013c850 msgr2=0x7f4b6014ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T23:48:41.362 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:41.375+0000 7f4b5ffff640 1 -- 172.21.15.57:0/2640779937 >> v2:172.21.15.80:3300/0 conn(0x7f4b6013c850 msgr2=0x7f4b6014ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T23:48:41.362 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:41.375+0000 7f4b5ffff640 1 --2- 172.21.15.57:0/2640779937 >> v2:172.21.15.80:3300/0 conn(0x7f4b6013c850 0x7f4b6014ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T23:48:41.362 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:41.375+0000 7f4b5ffff640 1 --2- 172.21.15.57:0/2640779937 >> v2:172.21.15.80:3300/0 conn(0x7f4b6013c850 0x7f4b6014ab60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T23:48:42.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:42.963+0000 7f4b5dffb640 1 -- 172.21.15.57:0/2640779937 >> v2:172.21.15.80:3300/0 conn(0x7f4b6013c850 msgr2=0x7f4b6014ab60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T23:48:42.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:42.963+0000 7f4b5dffb640 1 --2- 172.21.15.57:0/2640779937 >> v2:172.21.15.80:3300/0 conn(0x7f4b6013c850 0x7f4b6014ab60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:42.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:42.963+0000 7f4b5dffb640 1 -- 172.21.15.57:0/2640779937 >> v1:172.21.15.80:6789/0 conn(0x7f4b6013f950 legacy=0x7f4b6013c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:42.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:42.963+0000 7f4b5dffb640 1 -- 172.21.15.57:0/2640779937 >> v1:172.21.15.57:6789/0 conn(0x7f4b60073050 legacy=0x7f4b6013f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:42.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:42.963+0000 7f4b5dffb640 1 --2- 172.21.15.57:0/2640779937 >> v2:172.21.15.57:3300/0 conn(0x7f4b400036f0 0x7f4b40005b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:42.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:42.963+0000 7f4b5dffb640 1 --2- 172.21.15.57:0/2640779937 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4b40006190 0x7f4b40008650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:42.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:42.963+0000 7f4b5dffb640 1 -- 172.21.15.57:0/2640779937 --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 -- 0x7f4b60071780 con 0x7f4b400036f0 2024-04-25T23:48:42.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:42.963+0000 7f4b5dffb640 1 -- 172.21.15.57:0/2640779937 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f4b40008d70 con 0x7f4b40006190 2024-04-25T23:48:42.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:42.967+0000 7f4b5f7fe640 1 --2- 172.21.15.57:0/2640779937 >> v2:172.21.15.57:3300/0 conn(0x7f4b400036f0 0x7f4b40005b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:42.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:42.967+0000 7f4b56ffd640 1 --2- 172.21.15.57:0/2640779937 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4b40006190 0x7f4b40008650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:42.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:42.967+0000 7f4b5f7fe640 1 -- 172.21.15.57:0/2640779937 >> v2:172.21.15.57:3300/0 conn(0x7f4b400036f0 msgr2=0x7f4b40005b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T23:48:42.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:42.967+0000 7f4b5f7fe640 1 -- 172.21.15.57:0/2640779937 >> v2:172.21.15.57:3300/0 conn(0x7f4b400036f0 msgr2=0x7f4b40005b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T23:48:42.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:42.967+0000 7f4b5f7fe640 1 --2- 172.21.15.57:0/2640779937 >> v2:172.21.15.57:3300/0 conn(0x7f4b400036f0 0x7f4b40005b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T23:48:42.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:42.967+0000 7f4b56ffd640 1 -- 172.21.15.57:0/2640779937 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4b40006190 msgr2=0x7f4b40008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T23:48:42.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:42.967+0000 7f4b56ffd640 1 -- 172.21.15.57:0/2640779937 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4b40006190 msgr2=0x7f4b40008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T23:48:42.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:42.967+0000 7f4b56ffd640 1 --2- 172.21.15.57:0/2640779937 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4b40006190 0x7f4b40008650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T23:48:42.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:42.967+0000 7f4b5f7fe640 1 --2- 172.21.15.57:0/2640779937 >> v2:172.21.15.57:3300/0 conn(0x7f4b400036f0 0x7f4b40005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T23:48:42.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:42.967+0000 7f4b56ffd640 1 --2- 172.21.15.57:0/2640779937 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4b40006190 0x7f4b40008650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T23:48:43.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:43.171+0000 7f4b5f7fe640 1 --2- 172.21.15.57:0/2640779937 >> v2:172.21.15.57:3300/0 conn(0x7f4b400036f0 0x7f4b40005b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:43.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:43.171+0000 7f4b56ffd640 1 --2- 172.21.15.57:0/2640779937 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4b40006190 0x7f4b40008650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:43.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:43.171+0000 7f4b5f7fe640 1 -- 172.21.15.57:0/2640779937 >> v2:172.21.15.57:3300/0 conn(0x7f4b400036f0 msgr2=0x7f4b40005b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T23:48:43.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:43.171+0000 7f4b5f7fe640 1 -- 172.21.15.57:0/2640779937 >> v2:172.21.15.57:3300/0 conn(0x7f4b400036f0 msgr2=0x7f4b40005b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T23:48:43.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:43.171+0000 7f4b5f7fe640 1 --2- 172.21.15.57:0/2640779937 >> v2:172.21.15.57:3300/0 conn(0x7f4b400036f0 0x7f4b40005b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T23:48:43.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:43.171+0000 7f4b56ffd640 1 -- 172.21.15.57:0/2640779937 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4b40006190 msgr2=0x7f4b40008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T23:48:43.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:43.171+0000 7f4b56ffd640 1 -- 172.21.15.57:0/2640779937 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4b40006190 msgr2=0x7f4b40008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T23:48:43.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:43.171+0000 7f4b56ffd640 1 --2- 172.21.15.57:0/2640779937 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4b40006190 0x7f4b40008650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T23:48:43.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:43.171+0000 7f4b56ffd640 1 --2- 172.21.15.57:0/2640779937 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4b40006190 0x7f4b40008650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T23:48:43.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:43.171+0000 7f4b5f7fe640 1 --2- 172.21.15.57:0/2640779937 >> v2:172.21.15.57:3300/0 conn(0x7f4b400036f0 0x7f4b40005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T23:48:43.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:43.575+0000 7f4b5f7fe640 1 --2- 172.21.15.57:0/2640779937 >> v2:172.21.15.57:3300/0 conn(0x7f4b400036f0 0x7f4b40005b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:43.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:43.575+0000 7f4b56ffd640 1 --2- 172.21.15.57:0/2640779937 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4b40006190 0x7f4b40008650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:43.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:43.575+0000 7f4b56ffd640 1 -- 172.21.15.57:0/2640779937 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4b40006190 msgr2=0x7f4b40008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T23:48:43.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:43.575+0000 7f4b56ffd640 1 -- 172.21.15.57:0/2640779937 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4b40006190 msgr2=0x7f4b40008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T23:48:43.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:43.575+0000 7f4b56ffd640 1 --2- 172.21.15.57:0/2640779937 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4b40006190 0x7f4b40008650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T23:48:43.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:43.575+0000 7f4b5f7fe640 1 -- 172.21.15.57:0/2640779937 >> v2:172.21.15.57:3300/0 conn(0x7f4b400036f0 msgr2=0x7f4b40005b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T23:48:43.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:43.575+0000 7f4b5f7fe640 1 -- 172.21.15.57:0/2640779937 >> v2:172.21.15.57:3300/0 conn(0x7f4b400036f0 msgr2=0x7f4b40005b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T23:48:43.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:43.575+0000 7f4b5f7fe640 1 --2- 172.21.15.57:0/2640779937 >> v2:172.21.15.57:3300/0 conn(0x7f4b400036f0 0x7f4b40005b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T23:48:43.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:43.575+0000 7f4b56ffd640 1 --2- 172.21.15.57:0/2640779937 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4b40006190 0x7f4b40008650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T23:48:43.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:43.575+0000 7f4b5f7fe640 1 --2- 172.21.15.57:0/2640779937 >> v2:172.21.15.57:3300/0 conn(0x7f4b400036f0 0x7f4b40005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T23:48:44.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:44.379+0000 7f4b56ffd640 1 --2- 172.21.15.57:0/2640779937 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4b40006190 0x7f4b40008650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:44.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:44.379+0000 7f4b5f7fe640 1 --2- 172.21.15.57:0/2640779937 >> v2:172.21.15.57:3300/0 conn(0x7f4b400036f0 0x7f4b40005b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:44.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:44.379+0000 7f4b56ffd640 1 -- 172.21.15.57:0/2640779937 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4b40006190 msgr2=0x7f4b40008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T23:48:44.366 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:44.379+0000 7f4b56ffd640 1 -- 172.21.15.57:0/2640779937 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4b40006190 msgr2=0x7f4b40008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T23:48:44.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:44.379+0000 7f4b56ffd640 1 --2- 172.21.15.57:0/2640779937 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4b40006190 0x7f4b40008650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T23:48:44.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:44.379+0000 7f4b5f7fe640 1 -- 172.21.15.57:0/2640779937 >> v2:172.21.15.57:3300/0 conn(0x7f4b400036f0 msgr2=0x7f4b40005b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T23:48:44.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:44.379+0000 7f4b5f7fe640 1 -- 172.21.15.57:0/2640779937 >> v2:172.21.15.57:3300/0 conn(0x7f4b400036f0 msgr2=0x7f4b40005b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T23:48:44.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:44.379+0000 7f4b5f7fe640 1 --2- 172.21.15.57:0/2640779937 >> v2:172.21.15.57:3300/0 conn(0x7f4b400036f0 0x7f4b40005b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T23:48:44.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:44.379+0000 7f4b5f7fe640 1 --2- 172.21.15.57:0/2640779937 >> v2:172.21.15.57:3300/0 conn(0x7f4b400036f0 0x7f4b40005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T23:48:44.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:44.379+0000 7f4b56ffd640 1 --2- 172.21.15.57:0/2640779937 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4b40006190 0x7f4b40008650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T23:48:45.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:45.963+0000 7f4b5dffb640 1 -- 172.21.15.57:0/2640779937 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4b40006190 msgr2=0x7f4b40008650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T23:48:45.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:45.963+0000 7f4b5dffb640 1 --2- 172.21.15.57:0/2640779937 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4b40006190 0x7f4b40008650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:45.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:45.963+0000 7f4b5dffb640 1 -- 172.21.15.57:0/2640779937 >> v2:172.21.15.57:3300/0 conn(0x7f4b400036f0 msgr2=0x7f4b40005b50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T23:48:45.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:45.963+0000 7f4b5dffb640 1 --2- 172.21.15.57:0/2640779937 >> v2:172.21.15.57:3300/0 conn(0x7f4b400036f0 0x7f4b40005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:45.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:45.967+0000 7f4b5ffff640 1 -- 172.21.15.57:0/2640779937 reap_dead start 2024-04-25T23:48:45.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:45.967+0000 7f4b5dffb640 1 --2- 172.21.15.57:0/2640779937 >> v2:172.21.15.57:3300/0 conn(0x7f4b4000b480 0x7f4b4000d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:45.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:45.967+0000 7f4b5dffb640 1 --2- 172.21.15.57:0/2640779937 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4b40006640 0x7f4b4000de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:45.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:45.967+0000 7f4b5dffb640 1 --2- 172.21.15.57:0/2640779937 >> v2:172.21.15.80:3300/0 conn(0x7f4b4000e410 0x7f4b400108d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:45.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:45.967+0000 7f4b5dffb640 1 -- 172.21.15.57:0/2640779937 --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 -- 0x7f4b60071780 con 0x7f4b4000b480 2024-04-25T23:48:45.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:45.967+0000 7f4b5dffb640 1 -- 172.21.15.57:0/2640779937 --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f4b40008d70 con 0x7f4b4000e410 2024-04-25T23:48:45.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:45.967+0000 7f4b5dffb640 1 -- 172.21.15.57:0/2640779937 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f4b400110a0 con 0x7f4b40006640 2024-04-25T23:48:45.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:45.967+0000 7f4b56ffd640 1 --2- 172.21.15.57:0/2640779937 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4b40006640 0x7f4b4000de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:45.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:45.967+0000 7f4b5f7fe640 1 --2- 172.21.15.57:0/2640779937 >> v2:172.21.15.57:3300/0 conn(0x7f4b4000b480 0x7f4b4000d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:45.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:45.967+0000 7f4b5ffff640 1 --2- 172.21.15.57:0/2640779937 >> v2:172.21.15.80:3300/0 conn(0x7f4b4000e410 0x7f4b400108d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:45.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:45.967+0000 7f4b56ffd640 1 -- 172.21.15.57:0/2640779937 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4b40006640 msgr2=0x7f4b4000de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T23:48:45.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:45.967+0000 7f4b56ffd640 1 -- 172.21.15.57:0/2640779937 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4b40006640 msgr2=0x7f4b4000de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T23:48:45.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:45.967+0000 7f4b56ffd640 1 --2- 172.21.15.57:0/2640779937 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4b40006640 0x7f4b4000de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T23:48:45.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:45.967+0000 7f4b56ffd640 1 --2- 172.21.15.57:0/2640779937 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4b40006640 0x7f4b4000de50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T23:48:45.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:45.967+0000 7f4b5ffff640 1 -- 172.21.15.57:0/2640779937 >> v2:172.21.15.80:3300/0 conn(0x7f4b4000e410 msgr2=0x7f4b400108d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T23:48:45.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:45.967+0000 7f4b5ffff640 1 -- 172.21.15.57:0/2640779937 >> v2:172.21.15.80:3300/0 conn(0x7f4b4000e410 msgr2=0x7f4b400108d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T23:48:45.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:45.967+0000 7f4b5ffff640 1 --2- 172.21.15.57:0/2640779937 >> v2:172.21.15.80:3300/0 conn(0x7f4b4000e410 0x7f4b400108d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T23:48:45.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:45.967+0000 7f4b5f7fe640 1 -- 172.21.15.57:0/2640779937 >> v2:172.21.15.57:3300/0 conn(0x7f4b4000b480 msgr2=0x7f4b4000d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T23:48:45.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:45.967+0000 7f4b5f7fe640 1 -- 172.21.15.57:0/2640779937 >> v2:172.21.15.57:3300/0 conn(0x7f4b4000b480 msgr2=0x7f4b4000d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T23:48:45.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:45.967+0000 7f4b5ffff640 1 --2- 172.21.15.57:0/2640779937 >> v2:172.21.15.80:3300/0 conn(0x7f4b4000e410 0x7f4b400108d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T23:48:45.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:45.967+0000 7f4b5f7fe640 1 --2- 172.21.15.57:0/2640779937 >> v2:172.21.15.57:3300/0 conn(0x7f4b4000b480 0x7f4b4000d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T23:48:45.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:45.967+0000 7f4b5f7fe640 1 --2- 172.21.15.57:0/2640779937 >> v2:172.21.15.57:3300/0 conn(0x7f4b4000b480 0x7f4b4000d870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T23:48:46.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.171+0000 7f4b56ffd640 1 --2- 172.21.15.57:0/2640779937 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4b40006640 0x7f4b4000de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:46.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.171+0000 7f4b5f7fe640 1 --2- 172.21.15.57:0/2640779937 >> v2:172.21.15.57:3300/0 conn(0x7f4b4000b480 0x7f4b4000d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:46.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.171+0000 7f4b5ffff640 1 --2- 172.21.15.57:0/2640779937 >> v2:172.21.15.80:3300/0 conn(0x7f4b4000e410 0x7f4b400108d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:46.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.171+0000 7f4b56ffd640 1 -- 172.21.15.57:0/2640779937 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4b40006640 msgr2=0x7f4b4000de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T23:48:46.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.171+0000 7f4b56ffd640 1 -- 172.21.15.57:0/2640779937 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4b40006640 msgr2=0x7f4b4000de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T23:48:46.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.171+0000 7f4b56ffd640 1 --2- 172.21.15.57:0/2640779937 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4b40006640 0x7f4b4000de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T23:48:46.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.171+0000 7f4b5f7fe640 1 -- 172.21.15.57:0/2640779937 >> v2:172.21.15.57:3300/0 conn(0x7f4b4000b480 msgr2=0x7f4b4000d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T23:48:46.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.171+0000 7f4b56ffd640 1 --2- 172.21.15.57:0/2640779937 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4b40006640 0x7f4b4000de50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T23:48:46.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.171+0000 7f4b5f7fe640 1 -- 172.21.15.57:0/2640779937 >> v2:172.21.15.57:3300/0 conn(0x7f4b4000b480 msgr2=0x7f4b4000d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T23:48:46.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.171+0000 7f4b5f7fe640 1 --2- 172.21.15.57:0/2640779937 >> v2:172.21.15.57:3300/0 conn(0x7f4b4000b480 0x7f4b4000d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T23:48:46.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.171+0000 7f4b5ffff640 1 -- 172.21.15.57:0/2640779937 >> v2:172.21.15.80:3300/0 conn(0x7f4b4000e410 msgr2=0x7f4b400108d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T23:48:46.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.171+0000 7f4b5ffff640 1 -- 172.21.15.57:0/2640779937 >> v2:172.21.15.80:3300/0 conn(0x7f4b4000e410 msgr2=0x7f4b400108d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T23:48:46.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.171+0000 7f4b5ffff640 1 --2- 172.21.15.57:0/2640779937 >> v2:172.21.15.80:3300/0 conn(0x7f4b4000e410 0x7f4b400108d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T23:48:46.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.171+0000 7f4b5f7fe640 1 --2- 172.21.15.57:0/2640779937 >> v2:172.21.15.57:3300/0 conn(0x7f4b4000b480 0x7f4b4000d870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T23:48:46.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.171+0000 7f4b5ffff640 1 --2- 172.21.15.57:0/2640779937 >> v2:172.21.15.80:3300/0 conn(0x7f4b4000e410 0x7f4b400108d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T23:48:46.429 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-04-25T23:48:46.445+0000 7f625a234180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T23:48:46.491 INFO:tasks.ceph.mgr.y.smithi057.stderr:2024-04-25T23:48:46.503+0000 7f7104d7b180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T23:48:46.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.575+0000 7f4b5f7fe640 1 --2- 172.21.15.57:0/2640779937 >> v2:172.21.15.57:3300/0 conn(0x7f4b4000b480 0x7f4b4000d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:46.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.575+0000 7f4b56ffd640 1 --2- 172.21.15.57:0/2640779937 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4b40006640 0x7f4b4000de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:46.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.575+0000 7f4b5ffff640 1 --2- 172.21.15.57:0/2640779937 >> v2:172.21.15.80:3300/0 conn(0x7f4b4000e410 0x7f4b400108d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:46.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.575+0000 7f4b56ffd640 1 -- 172.21.15.57:0/2640779937 >> v2:172.21.15.80:3300/0 conn(0x7f4b4000e410 msgr2=0x7f4b400108d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:48:46.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.575+0000 7f4b56ffd640 1 --2- 172.21.15.57:0/2640779937 >> v2:172.21.15.80:3300/0 conn(0x7f4b4000e410 0x7f4b400108d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:46.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.575+0000 7f4b56ffd640 1 -- 172.21.15.57:0/2640779937 >> v2:172.21.15.57:3300/0 conn(0x7f4b4000b480 msgr2=0x7f4b4000d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:48:46.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.575+0000 7f4b56ffd640 1 --2- 172.21.15.57:0/2640779937 >> v2:172.21.15.57:3300/0 conn(0x7f4b4000b480 0x7f4b4000d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:46.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.575+0000 7f4b56ffd640 1 -- 172.21.15.57:0/2640779937 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b6013cd80 con 0x7f4b40006640 2024-04-25T23:48:46.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.579+0000 7f4b56ffd640 1 --2- 172.21.15.57:0/2640779937 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4b40006640 0x7f4b4000de50 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4b4000e3e0 tx=0x7f4b4801d4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1870b7bbbe9aab11 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:48:46.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.579+0000 7f4b5effd640 1 -- 172.21.15.57:0/2640779937 <== mon.2 v2:172.21.15.57:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b480af440 con 0x7f4b40006640 2024-04-25T23:48:46.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.579+0000 7f4b5effd640 1 -- 172.21.15.57:0/2640779937 <== mon.2 v2:172.21.15.57:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b480af5e0 con 0x7f4b40006640 2024-04-25T23:48:46.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.579+0000 7f4b5effd640 1 -- 172.21.15.57:0/2640779937 <== mon.2 v2:172.21.15.57:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b480af9f0 con 0x7f4b40006640 2024-04-25T23:48:46.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.583+0000 7f4b65d2d640 1 -- 172.21.15.57:0/2640779937 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4b40006640 msgr2=0x7f4b4000de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:46.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.583+0000 7f4b65d2d640 1 --2- 172.21.15.57:0/2640779937 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4b40006640 0x7f4b4000de50 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4b4000e3e0 tx=0x7f4b4801d4a0 comp rx=0 tx=0).stop 2024-04-25T23:48:46.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.583+0000 7f4b65d2d640 1 -- 172.21.15.57:0/2640779937 shutdown_connections 2024-04-25T23:48:46.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.583+0000 7f4b65d2d640 1 --2- 172.21.15.57:0/2640779937 >> v2:172.21.15.80:3300/0 conn(0x7f4b4000e410 0x7f4b400108d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:46.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.583+0000 7f4b65d2d640 1 --2- 172.21.15.57:0/2640779937 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4b40006640 0x7f4b4000de50 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:46.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.583+0000 7f4b65d2d640 1 --2- 172.21.15.57:0/2640779937 >> v2:172.21.15.57:3300/0 conn(0x7f4b4000b480 0x7f4b4000d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:46.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.583+0000 7f4b65d2d640 1 -- 172.21.15.57:0/2640779937 >> 172.21.15.57:0/2640779937 conn(0x7f4b600b5320 msgr2=0x7f4b600b5730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:48:46.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.583+0000 7f4b65d2d640 1 -- 172.21.15.57:0/2640779937 shutdown_connections 2024-04-25T23:48:46.570 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.583+0000 7f4b65d2d640 1 -- 172.21.15.57:0/2640779937 wait complete. 2024-04-25T23:48:46.572 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.583+0000 7f4b65d2d640 1 Processor -- start 2024-04-25T23:48:46.572 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.583+0000 7f4b65d2d640 1 -- start start 2024-04-25T23:48:46.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.587+0000 7f4b65d2d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b40006640 0x7f4b600b3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:46.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.587+0000 7f4b65d2d640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b4000b480 0x7f4b601cbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:46.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.587+0000 7f4b65d2d640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4b4000e410 0x7f4b601ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:46.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.587+0000 7f4b65d2d640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f4b60070a80 con 0x7f4b4000b480 2024-04-25T23:48:46.575 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.587+0000 7f4b65d2d640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f4b60070900 con 0x7f4b40006640 2024-04-25T23:48:46.576 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.587+0000 7f4b65d2d640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f4b60142fd0 con 0x7f4b4000e410 2024-04-25T23:48:46.576 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.587+0000 7f4b56ffd640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b4000b480 0x7f4b601cbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:46.576 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.587+0000 7f4b56ffd640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b4000b480 0x7f4b601cbfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44462/0 (socket says 172.21.15.57:44462) 2024-04-25T23:48:46.576 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.587+0000 7f4b56ffd640 1 -- 172.21.15.57:0/3981063801 learned_addr learned my addr 172.21.15.57:0/3981063801 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:48:46.576 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.587+0000 7f4b5ffff640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4b4000e410 0x7f4b601ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:46.576 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.587+0000 7f4b5f7fe640 1 --2- 172.21.15.57:0/3981063801 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b40006640 0x7f4b600b3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:46.576 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.587+0000 7f4b56ffd640 1 -- 172.21.15.57:0/3981063801 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4b4000e410 msgr2=0x7f4b601ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:46.576 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.587+0000 7f4b56ffd640 1 --2- 172.21.15.57:0/3981063801 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4b4000e410 0x7f4b601ccc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:46.576 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.587+0000 7f4b56ffd640 1 -- 172.21.15.57:0/3981063801 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b40006640 msgr2=0x7f4b600b3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:46.576 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.587+0000 7f4b56ffd640 1 --2- 172.21.15.57:0/3981063801 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b40006640 0x7f4b600b3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:46.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.587+0000 7f4b56ffd640 1 -- 172.21.15.57:0/3981063801 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b601ce340 con 0x7f4b4000b480 2024-04-25T23:48:46.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.587+0000 7f4b56ffd640 1 --2- 172.21.15.57:0/3981063801 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b4000b480 0x7f4b601cbfc0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4b4801bb50 tx=0x7f4b4801bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:48:46.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.587+0000 7f4b5cff9640 1 -- 172.21.15.57:0/3981063801 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b480bc020 con 0x7f4b4000b480 2024-04-25T23:48:46.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.587+0000 7f4b5cff9640 1 -- 172.21.15.57:0/3981063801 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b480b8610 con 0x7f4b4000b480 2024-04-25T23:48:46.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.587+0000 7f4b65d2d640 1 -- 172.21.15.57:0/3981063801 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b601ce620 con 0x7f4b4000b480 2024-04-25T23:48:46.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.587+0000 7f4b5cff9640 1 -- 172.21.15.57:0/3981063801 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b480b4040 con 0x7f4b4000b480 2024-04-25T23:48:46.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.587+0000 7f4b65d2d640 1 -- 172.21.15.57:0/3981063801 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b601da070 con 0x7f4b4000b480 2024-04-25T23:48:46.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.587+0000 7f4b5cff9640 1 -- 172.21.15.57:0/3981063801 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4b480bf020 con 0x7f4b4000b480 2024-04-25T23:48:46.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.587+0000 7f4b5cff9640 1 -- 172.21.15.57:0/3981063801 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4b480c14f0 con 0x7f4b4000b480 2024-04-25T23:48:46.578 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.587+0000 7f4b65d2d640 1 -- 172.21.15.57:0/3981063801 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b60068210 con 0x7f4b4000b480 2024-04-25T23:48:46.580 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.591+0000 7f4b5cff9640 1 -- 172.21.15.57:0/3981063801 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f4b480c6030 con 0x7f4b4000b480 2024-04-25T23:48:46.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.655+0000 7f4b65d2d640 1 -- 172.21.15.57:0/3981063801 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f4b601cc770 con 0x7f4b4000b480 2024-04-25T23:48:46.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.667+0000 7f4b5cff9640 1 -- 172.21.15.57:0/3981063801 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f4b48008dc0 con 0x7f4b4000b480 2024-04-25T23:48:46.654 INFO:teuthology.orchestra.run.smithi057.stderr:adjusted tunables profile to default 2024-04-25T23:48:46.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.667+0000 7f4b65d2d640 1 -- 172.21.15.57:0/3981063801 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b4000b480 msgr2=0x7f4b601cbfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:46.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.667+0000 7f4b65d2d640 1 --2- 172.21.15.57:0/3981063801 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b4000b480 0x7f4b601cbfc0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4b4801bb50 tx=0x7f4b4801bce0 comp rx=0 tx=0).stop 2024-04-25T23:48:46.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.667+0000 7f4b65d2d640 1 -- 172.21.15.57:0/3981063801 shutdown_connections 2024-04-25T23:48:46.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.667+0000 7f4b65d2d640 1 --2- 172.21.15.57:0/3981063801 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4b4000e410 0x7f4b601ccc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:46.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.667+0000 7f4b65d2d640 1 --2- 172.21.15.57:0/3981063801 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b4000b480 0x7f4b601cbfc0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:46.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.667+0000 7f4b65d2d640 1 --2- 172.21.15.57:0/3981063801 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b40006640 0x7f4b600b3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:46.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.667+0000 7f4b65d2d640 1 -- 172.21.15.57:0/3981063801 >> 172.21.15.57:0/3981063801 conn(0x7f4b600b5320 msgr2=0x7f4b6013a420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:48:46.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.667+0000 7f4b65d2d640 1 -- 172.21.15.57:0/3981063801 shutdown_connections 2024-04-25T23:48:46.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:46.667+0000 7f4b65d2d640 1 -- 172.21.15.57:0/3981063801 wait complete. 2024-04-25T23:48:46.673 INFO:tasks.ceph:check_enable_crimson: False 2024-04-25T23:48:46.674 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-25T23:48:46.674 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-04-25T23:48:46.674 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-25T23:48:46.685 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-04-25T23:48:46.685 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-25T23:48:46.739 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-04-25T23:48:46.739 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-25T23:48:46.791 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-04-25T23:48:46.791 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-25T23:48:46.843 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T23:48:46.843 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-25T23:48:46.857 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T23:48:46.857 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-25T23:48:46.912 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T23:48:46.912 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-25T23:48:46.931 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-04-25T23:48:46.945+0000 7f625a234180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T23:48:46.942 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T23:48:46.942 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-25T23:48:46.995 DEBUG:teuthology.orchestra.run.smithi080:> sudo ceph --cluster ceph osd new 5986c317-8828-4568-af68-a8ae96690c51 0 2024-04-25T23:48:47.016 INFO:tasks.ceph.mgr.y.smithi057.stderr:2024-04-25T23:48:47.027+0000 7f7104d7b180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T23:48:47.125 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-04-25T23:48:47.141+0000 7f625a234180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T23:48:47.155 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.169+0000 7ff6fad65640 1 Processor -- start 2024-04-25T23:48:47.155 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.169+0000 7ff6fad65640 1 -- start start 2024-04-25T23:48:47.156 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.169+0000 7ff6fad65640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff6f4072b60 0x7ff6f414acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:47.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.169+0000 7ff6fad65640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6f4071d50 con 0x7ff6f413f490 2024-04-25T23:48:47.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.169+0000 7ff6fad65640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6f40b0ee0 con 0x7ff6f40b5200 2024-04-25T23:48:47.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.169+0000 7ff6fad65640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7ff6f40b55e0 con 0x7ff6f4072b60 2024-04-25T23:48:47.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.169+0000 7ff6f8b01640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7ff6f40b5200 0x7ff6f4072450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:49028/0 (socket says 172.21.15.80:49028) 2024-04-25T23:48:47.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.169+0000 7ff6f8b01640 1 -- 172.21.15.80:0/1724543432 learned_addr learned my addr 172.21.15.80:0/1724543432 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T23:48:47.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.169+0000 7ff6f9302640 1 --2- 172.21.15.80:0/1724543432 >> v2:172.21.15.80:3300/0 conn(0x7ff6f4072b60 0x7ff6f414acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:47.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.169+0000 7ff6f9302640 1 -- 172.21.15.80:0/1724543432 >> v1:172.21.15.80:6789/0 conn(0x7ff6f40b5200 legacy=0x7ff6f4072450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:47.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.169+0000 7ff6f9302640 1 -- 172.21.15.80:0/1724543432 >> v1:172.21.15.57:6789/0 conn(0x7ff6f413f490 legacy=0x7ff6f413c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:48:47.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.169+0000 7ff6f37fe640 1 -- 172.21.15.80:0/1724543432 <== mon.1 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 428669690 0 0) 0x7ff6f40b0ee0 con 0x7ff6f40b5200 2024-04-25T23:48:47.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.169+0000 7ff6f9302640 1 -- 172.21.15.80:0/1724543432 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6f4070a20 con 0x7ff6f4072b60 2024-04-25T23:48:47.160 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.173+0000 7ff6f9302640 1 --2- 172.21.15.80:0/1724543432 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6f4072b60 0x7ff6f414acf0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff6e40097f0 tx=0x7ff6e401c370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c28697a51a88b72 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:48:47.160 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.173+0000 7ff6f37fe640 1 -- 172.21.15.80:0/1724543432 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e40b0070 con 0x7ff6f4072b60 2024-04-25T23:48:47.160 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.173+0000 7ff6f37fe640 1 -- 172.21.15.80:0/1724543432 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6e40ab600 con 0x7ff6f4072b60 2024-04-25T23:48:47.160 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.173+0000 7ff6f37fe640 1 -- 172.21.15.80:0/1724543432 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e40ab9d0 con 0x7ff6f4072b60 2024-04-25T23:48:47.161 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.173+0000 7ff6fad65640 1 -- 172.21.15.80:0/1724543432 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6f4072b60 msgr2=0x7ff6f414acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:47.161 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.173+0000 7ff6fad65640 1 --2- 172.21.15.80:0/1724543432 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6f4072b60 0x7ff6f414acf0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff6e40097f0 tx=0x7ff6e401c370 comp rx=0 tx=0).stop 2024-04-25T23:48:47.161 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.173+0000 7ff6fad65640 1 -- 172.21.15.80:0/1724543432 shutdown_connections 2024-04-25T23:48:47.161 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.173+0000 7ff6fad65640 1 --2- 172.21.15.80:0/1724543432 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6f4072b60 0x7ff6f414acf0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:47.161 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.173+0000 7ff6fad65640 1 -- 172.21.15.80:0/1724543432 >> 172.21.15.80:0/1724543432 conn(0x7ff6f406b960 msgr2=0x7ff6f40b4890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:48:47.161 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.173+0000 7ff6fad65640 1 -- 172.21.15.80:0/1724543432 shutdown_connections 2024-04-25T23:48:47.161 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.173+0000 7ff6fad65640 1 -- 172.21.15.80:0/1724543432 wait complete. 2024-04-25T23:48:47.164 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.177+0000 7ff6fad65640 1 Processor -- start 2024-04-25T23:48:47.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.177+0000 7ff6fad65640 1 -- start start 2024-04-25T23:48:47.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.177+0000 7ff6fad65640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6f4072b60 0x7ff6f413c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:47.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.177+0000 7ff6fad65640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7ff6f40b5200 0x7ff6f4141fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:47.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.177+0000 7ff6fad65640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6f413f490 0x7ff6f4141580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:47.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.177+0000 7ff6fad65640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f414dc60 con 0x7ff6f413f490 2024-04-25T23:48:47.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.177+0000 7ff6fad65640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f40aecb0 con 0x7ff6f4072b60 2024-04-25T23:48:47.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.177+0000 7ff6fad65640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7ff6f414dde0 con 0x7ff6f40b5200 2024-04-25T23:48:47.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.177+0000 7ff6f8b01640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6f4072b60 0x7ff6f413c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:47.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.177+0000 7ff6f8b01640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6f4072b60 0x7ff6f413c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:59084/0 (socket says 172.21.15.80:59084) 2024-04-25T23:48:47.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.177+0000 7ff6f8b01640 1 -- 172.21.15.80:0/42166171 learned_addr learned my addr 172.21.15.80:0/42166171 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T23:48:47.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.177+0000 7ff6f3fff640 1 --2- 172.21.15.80:0/42166171 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7ff6f40b5200 0x7ff6f4141fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:47.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.177+0000 7ff6f9302640 1 --2- 172.21.15.80:0/42166171 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6f413f490 0x7ff6f4141580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:47.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.177+0000 7ff6f8b01640 1 -- 172.21.15.80:0/42166171 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7ff6f40b5200 msgr2=0x7ff6f4141fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:47.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.177+0000 7ff6f8b01640 1 --2- 172.21.15.80:0/42166171 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7ff6f40b5200 0x7ff6f4141fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:47.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.177+0000 7ff6f8b01640 1 -- 172.21.15.80:0/42166171 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6f413f490 msgr2=0x7ff6f4141580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:47.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.177+0000 7ff6f8b01640 1 --2- 172.21.15.80:0/42166171 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6f413f490 0x7ff6f4141580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:47.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.177+0000 7ff6f8b01640 1 -- 172.21.15.80:0/42166171 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6f41444e0 con 0x7ff6f4072b60 2024-04-25T23:48:47.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.177+0000 7ff6f3fff640 1 --2- 172.21.15.80:0/42166171 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7ff6f40b5200 0x7ff6f4141fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T23:48:47.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.177+0000 7ff6f9302640 1 --2- 172.21.15.80:0/42166171 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6f413f490 0x7ff6f4141580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T23:48:47.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.177+0000 7ff6f8b01640 1 --2- 172.21.15.80:0/42166171 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6f4072b60 0x7ff6f413c410 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec001570 tx=0x7ff6ec00cd20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:48:47.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.177+0000 7ff6f17fa640 1 -- 172.21.15.80:0/42166171 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ec018070 con 0x7ff6f4072b60 2024-04-25T23:48:47.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.177+0000 7ff6f17fa640 1 -- 172.21.15.80:0/42166171 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6ec00a640 con 0x7ff6f4072b60 2024-04-25T23:48:47.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.177+0000 7ff6fad65640 1 -- 172.21.15.80:0/42166171 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6f41447c0 con 0x7ff6f4072b60 2024-04-25T23:48:47.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.177+0000 7ff6f17fa640 1 -- 172.21.15.80:0/42166171 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ec00aa30 con 0x7ff6f4072b60 2024-04-25T23:48:47.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.177+0000 7ff6fad65640 1 -- 172.21.15.80:0/42166171 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6f41de350 con 0x7ff6f4072b60 2024-04-25T23:48:47.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.177+0000 7ff6f17fa640 1 -- 172.21.15.80:0/42166171 <== mon.1 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff6ec01b070 con 0x7ff6f4072b60 2024-04-25T23:48:47.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.177+0000 7ff6f17fa640 1 -- 172.21.15.80:0/42166171 <== mon.1 v2:172.21.15.80:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7ff6ec01d780 con 0x7ff6f4072b60 2024-04-25T23:48:47.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.177+0000 7ff6fad65640 1 -- 172.21.15.80:0/42166171 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6f4068210 con 0x7ff6f4072b60 2024-04-25T23:48:47.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.193+0000 7ff6f17fa640 1 -- 172.21.15.80:0/42166171 <== mon.1 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7ff6ec00ac70 con 0x7ff6f4072b60 2024-04-25T23:48:47.230 INFO:tasks.ceph.mgr.y.smithi057.stderr:2024-04-25T23:48:47.243+0000 7f7104d7b180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T23:48:47.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.261+0000 7ff6fad65640 1 -- 172.21.15.80:0/42166171 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd new", "uuid": "5986c317-8828-4568-af68-a8ae96690c51", "id": 0} v 0) -- 0x7ff6f4142850 con 0x7ff6f4072b60 2024-04-25T23:48:47.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.265+0000 7ff6f17fa640 1 -- 172.21.15.80:0/42166171 <== mon.1 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5986c317-8828-4568-af68-a8ae96690c51", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7ff6ec01d9d0 con 0x7ff6f4072b60 2024-04-25T23:48:47.253 INFO:teuthology.orchestra.run.smithi080.stdout:0 2024-04-25T23:48:47.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.269+0000 7ff6fad65640 1 -- 172.21.15.80:0/42166171 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6f4072b60 msgr2=0x7ff6f413c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:47.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.269+0000 7ff6fad65640 1 --2- 172.21.15.80:0/42166171 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6f4072b60 0x7ff6f413c410 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec001570 tx=0x7ff6ec00cd20 comp rx=0 tx=0).stop 2024-04-25T23:48:47.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.269+0000 7ff6fad65640 1 -- 172.21.15.80:0/42166171 shutdown_connections 2024-04-25T23:48:47.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.269+0000 7ff6fad65640 1 --2- 172.21.15.80:0/42166171 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6f413f490 0x7ff6f4141580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:47.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.269+0000 7ff6fad65640 1 --2- 172.21.15.80:0/42166171 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7ff6f40b5200 0x7ff6f4141fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:47.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.269+0000 7ff6fad65640 1 --2- 172.21.15.80:0/42166171 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6f4072b60 0x7ff6f413c410 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:47.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.269+0000 7ff6fad65640 1 -- 172.21.15.80:0/42166171 >> 172.21.15.80:0/42166171 conn(0x7ff6f406b960 msgr2=0x7ff6f40b2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:48:47.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.269+0000 7ff6fad65640 1 -- 172.21.15.80:0/42166171 shutdown_connections 2024-04-25T23:48:47.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.269+0000 7ff6fad65640 1 -- 172.21.15.80:0/42166171 wait complete. 2024-04-25T23:48:47.269 DEBUG:teuthology.orchestra.run.smithi080:> sudo ceph --cluster ceph osd new 1aa157dd-52fe-4bb3-9f63-4db493de5333 1 2024-04-25T23:48:47.380 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.393+0000 7fc68d348640 1 Processor -- start 2024-04-25T23:48:47.380 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.393+0000 7fc68d348640 1 -- start start 2024-04-25T23:48:47.382 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.397+0000 7fc68d348640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fc688073050 0x7fc68813f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:47.382 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.397+0000 7fc68d348640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc68813f780 0x7fc68813c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:47.382 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.397+0000 7fc68d348640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc688072610 con 0x7fc68813c680 2024-04-25T23:48:47.382 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.397+0000 7fc68d348640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fc6880728f0 con 0x7fc68813f780 2024-04-25T23:48:47.382 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.397+0000 7fc68d348640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7fc688071780 con 0x7fc688073050 2024-04-25T23:48:47.382 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.397+0000 7fc6867fc640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc68813f780 0x7fc68813c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:47.382 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.397+0000 7fc6877fe640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fc68813c680 0x7fc6881427c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:49032/0 (socket says 172.21.15.80:49032) 2024-04-25T23:48:47.382 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.397+0000 7fc6877fe640 1 -- 172.21.15.80:0/2656334195 learned_addr learned my addr 172.21.15.80:0/2656334195 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T23:48:47.382 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.397+0000 7fc686ffd640 1 --2- 172.21.15.80:0/2656334195 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fc688073050 0x7fc68813f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:47.382 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.397+0000 7fc6867fc640 1 -- 172.21.15.80:0/2656334195 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fc688073050 msgr2=0x7fc68813f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:48:47.383 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.397+0000 7fc6867fc640 1 --2- 172.21.15.80:0/2656334195 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fc688073050 0x7fc68813f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:47.383 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.397+0000 7fc6867fc640 1 -- 172.21.15.80:0/2656334195 >> v1:172.21.15.80:6789/0 conn(0x7fc68813c680 legacy=0x7fc6881427c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:47.383 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.397+0000 7fc6867fc640 1 -- 172.21.15.80:0/2656334195 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc68813cd80 con 0x7fc68813f780 2024-04-25T23:48:47.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.397+0000 7fc6867fc640 1 --2- 172.21.15.80:0/2656334195 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc68813f780 0x7fc68813c140 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc67c0097c0 tx=0x7fc67c01c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67f03b8979ea451b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:48:47.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.397+0000 7fc685ffb640 1 -- 172.21.15.80:0/2656334195 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67c0b0070 con 0x7fc68813f780 2024-04-25T23:48:47.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.397+0000 7fc685ffb640 1 -- 172.21.15.80:0/2656334195 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc67c0ab600 con 0x7fc68813f780 2024-04-25T23:48:47.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.397+0000 7fc685ffb640 1 -- 172.21.15.80:0/2656334195 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67c0ab9d0 con 0x7fc68813f780 2024-04-25T23:48:47.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.397+0000 7fc68d348640 1 -- 172.21.15.80:0/2656334195 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc68813f780 msgr2=0x7fc68813c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:47.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.397+0000 7fc68d348640 1 --2- 172.21.15.80:0/2656334195 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc68813f780 0x7fc68813c140 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc67c0097c0 tx=0x7fc67c01c480 comp rx=0 tx=0).stop 2024-04-25T23:48:47.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.397+0000 7fc68d348640 1 -- 172.21.15.80:0/2656334195 shutdown_connections 2024-04-25T23:48:47.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.397+0000 7fc68d348640 1 --2- 172.21.15.80:0/2656334195 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc68813f780 0x7fc68813c140 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:47.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.397+0000 7fc68d348640 1 --2- 172.21.15.80:0/2656334195 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fc688073050 0x7fc68813f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:47.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.397+0000 7fc68d348640 1 -- 172.21.15.80:0/2656334195 >> 172.21.15.80:0/2656334195 conn(0x7fc6880b5320 msgr2=0x7fc6880b5730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:48:47.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.397+0000 7fc68d348640 1 -- 172.21.15.80:0/2656334195 shutdown_connections 2024-04-25T23:48:47.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.397+0000 7fc68d348640 1 -- 172.21.15.80:0/2656334195 wait complete. 2024-04-25T23:48:47.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.401+0000 7fc68d348640 1 Processor -- start 2024-04-25T23:48:47.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.401+0000 7fc68d348640 1 -- start start 2024-04-25T23:48:47.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.401+0000 7fc68d348640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc688073050 0x7fc6880ad660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:47.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.401+0000 7fc68d348640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc68813c680 0x7fc6880adf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:47.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.401+0000 7fc68d348640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fc68813f780 0x7fc6880ae470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:47.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.401+0000 7fc68d348640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fc68814ccf0 con 0x7fc68813c680 2024-04-25T23:48:47.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.401+0000 7fc68d348640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc68814cb70 con 0x7fc688073050 2024-04-25T23:48:47.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.401+0000 7fc68d348640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7fc68814ce70 con 0x7fc68813f780 2024-04-25T23:48:47.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.401+0000 7fc686ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc688073050 0x7fc6880ad660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:47.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.401+0000 7fc686ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc688073050 0x7fc6880ad660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:59102/0 (socket says 172.21.15.80:59102) 2024-04-25T23:48:47.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.401+0000 7fc686ffd640 1 -- 172.21.15.80:0/4158772570 learned_addr learned my addr 172.21.15.80:0/4158772570 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T23:48:47.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.401+0000 7fc6877fe640 1 --2- 172.21.15.80:0/4158772570 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fc68813f780 0x7fc6880ae470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:47.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.401+0000 7fc6867fc640 1 --2- 172.21.15.80:0/4158772570 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc68813c680 0x7fc6880adf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:47.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.401+0000 7fc686ffd640 1 -- 172.21.15.80:0/4158772570 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fc68813f780 msgr2=0x7fc6880ae470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:47.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.401+0000 7fc686ffd640 1 --2- 172.21.15.80:0/4158772570 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fc68813f780 0x7fc6880ae470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:47.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.401+0000 7fc686ffd640 1 -- 172.21.15.80:0/4158772570 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc68813c680 msgr2=0x7fc6880adf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:47.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.401+0000 7fc686ffd640 1 --2- 172.21.15.80:0/4158772570 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc68813c680 0x7fc6880adf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:47.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.401+0000 7fc686ffd640 1 -- 172.21.15.80:0/4158772570 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6880ac070 con 0x7fc688073050 2024-04-25T23:48:47.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.401+0000 7fc686ffd640 1 --2- 172.21.15.80:0/4158772570 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc688073050 0x7fc6880ad660 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc67800d800 tx=0x7fc67800dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:48:47.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.401+0000 7fc66f7fe640 1 -- 172.21.15.80:0/4158772570 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67800b6c0 con 0x7fc688073050 2024-04-25T23:48:47.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.401+0000 7fc66f7fe640 1 -- 172.21.15.80:0/4158772570 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc678019070 con 0x7fc688073050 2024-04-25T23:48:47.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.401+0000 7fc68d348640 1 -- 172.21.15.80:0/4158772570 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6880af4a0 con 0x7fc688073050 2024-04-25T23:48:47.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.401+0000 7fc66f7fe640 1 -- 172.21.15.80:0/4158772570 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67800ba60 con 0x7fc688073050 2024-04-25T23:48:47.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.401+0000 7fc68d348640 1 -- 172.21.15.80:0/4158772570 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc68814a620 con 0x7fc688073050 2024-04-25T23:48:47.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.401+0000 7fc66f7fe640 1 -- 172.21.15.80:0/4158772570 <== mon.1 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc67800bc00 con 0x7fc688073050 2024-04-25T23:48:47.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.401+0000 7fc66f7fe640 1 -- 172.21.15.80:0/4158772570 <== mon.1 v2:172.21.15.80:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7fc6780149a0 con 0x7fc688073050 2024-04-25T23:48:47.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.401+0000 7fc68d348640 1 -- 172.21.15.80:0/4158772570 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc688068210 con 0x7fc688073050 2024-04-25T23:48:47.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.405+0000 7fc66f7fe640 1 -- 172.21.15.80:0/4158772570 <== mon.1 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fc678014d00 con 0x7fc688073050 2024-04-25T23:48:47.458 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.473+0000 7fc68d348640 1 -- 172.21.15.80:0/4158772570 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd new", "uuid": "1aa157dd-52fe-4bb3-9f63-4db493de5333", "id": 1} v 0) -- 0x7fc688138ca0 con 0x7fc688073050 2024-04-25T23:48:47.470 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.485+0000 7fc66f7fe640 1 -- 172.21.15.80:0/4158772570 <== mon.1 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1aa157dd-52fe-4bb3-9f63-4db493de5333", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7fc67801e530 con 0x7fc688073050 2024-04-25T23:48:47.470 INFO:teuthology.orchestra.run.smithi080.stdout:1 2024-04-25T23:48:47.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.485+0000 7fc68d348640 1 -- 172.21.15.80:0/4158772570 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc688073050 msgr2=0x7fc6880ad660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:47.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.485+0000 7fc68d348640 1 --2- 172.21.15.80:0/4158772570 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc688073050 0x7fc6880ad660 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc67800d800 tx=0x7fc67800dcc0 comp rx=0 tx=0).stop 2024-04-25T23:48:47.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.485+0000 7fc68d348640 1 -- 172.21.15.80:0/4158772570 shutdown_connections 2024-04-25T23:48:47.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.485+0000 7fc68d348640 1 --2- 172.21.15.80:0/4158772570 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fc68813f780 0x7fc6880ae470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:47.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.485+0000 7fc68d348640 1 --2- 172.21.15.80:0/4158772570 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc68813c680 0x7fc6880adf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:47.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.485+0000 7fc68d348640 1 --2- 172.21.15.80:0/4158772570 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc688073050 0x7fc6880ad660 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:47.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.485+0000 7fc68d348640 1 -- 172.21.15.80:0/4158772570 >> 172.21.15.80:0/4158772570 conn(0x7fc6880b5320 msgr2=0x7fc6880751d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:48:47.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.485+0000 7fc68d348640 1 -- 172.21.15.80:0/4158772570 shutdown_connections 2024-04-25T23:48:47.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.485+0000 7fc68d348640 1 -- 172.21.15.80:0/4158772570 wait complete. 2024-04-25T23:48:47.487 DEBUG:teuthology.orchestra.run.smithi080:> sudo ceph --cluster ceph osd new 1391be38-25ac-43f0-8090-15adddc6d69d 2 2024-04-25T23:48:47.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.613+0000 7f0282df1640 1 Processor -- start 2024-04-25T23:48:47.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.613+0000 7f0282df1640 1 -- start start 2024-04-25T23:48:47.600 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.613+0000 7f0282df1640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f027c072060 0x7f027c144240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:47.600 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.613+0000 7f0282df1640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f027c0a95e0 con 0x7f027c0b1d80 2024-04-25T23:48:47.600 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.613+0000 7f0282df1640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f027c0726d0 con 0x7f027c0b2730 2024-04-25T23:48:47.600 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.613+0000 7f0282df1640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f027c0b0890 con 0x7f027c072060 2024-04-25T23:48:47.600 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.613+0000 7f027bfff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f027c0b2730 0x7f027c143b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:49038/0 (socket says 172.21.15.80:49038) 2024-04-25T23:48:47.600 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.613+0000 7f027bfff640 1 -- 172.21.15.80:0/3109304293 learned_addr learned my addr 172.21.15.80:0/3109304293 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T23:48:47.600 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.613+0000 7f028138e640 1 --2- 172.21.15.80:0/3109304293 >> v2:172.21.15.80:3300/0 conn(0x7f027c072060 0x7f027c144240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:47.600 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.613+0000 7f028138e640 1 -- 172.21.15.80:0/3109304293 >> v1:172.21.15.80:6789/0 conn(0x7f027c0b2730 legacy=0x7f027c143b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:47.600 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.613+0000 7f028138e640 1 -- 172.21.15.80:0/3109304293 >> v1:172.21.15.57:6789/0 conn(0x7f027c0b1d80 legacy=0x7f027c13a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:47.601 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.613+0000 7f028138e640 1 -- 172.21.15.80:0/3109304293 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f027c13a7f0 con 0x7f027c072060 2024-04-25T23:48:47.602 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.617+0000 7f028138e640 1 --2- 172.21.15.80:0/3109304293 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f027c072060 0x7f027c144240 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f027c13aa20 tx=0x7f027401c930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8aefbab855dabce4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:48:47.602 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.617+0000 7f027b7fe640 1 -- 172.21.15.80:0/3109304293 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02740b2070 con 0x7f027c072060 2024-04-25T23:48:47.603 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.617+0000 7f027b7fe640 1 -- 172.21.15.80:0/3109304293 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02740a57f0 con 0x7f027c072060 2024-04-25T23:48:47.603 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.617+0000 7f027b7fe640 1 -- 172.21.15.80:0/3109304293 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02740a5bc0 con 0x7f027c072060 2024-04-25T23:48:47.603 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.617+0000 7f0282df1640 1 -- 172.21.15.80:0/3109304293 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f027c072060 msgr2=0x7f027c144240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:47.603 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.617+0000 7f0282df1640 1 --2- 172.21.15.80:0/3109304293 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f027c072060 0x7f027c144240 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f027c13aa20 tx=0x7f027401c930 comp rx=0 tx=0).stop 2024-04-25T23:48:47.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.617+0000 7f0282df1640 1 -- 172.21.15.80:0/3109304293 shutdown_connections 2024-04-25T23:48:47.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.617+0000 7f0282df1640 1 --2- 172.21.15.80:0/3109304293 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f027c072060 0x7f027c144240 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:47.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.617+0000 7f0282df1640 1 -- 172.21.15.80:0/3109304293 >> 172.21.15.80:0/3109304293 conn(0x7f027c0b2c10 msgr2=0x7f027c0b3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:48:47.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.617+0000 7f0282df1640 1 -- 172.21.15.80:0/3109304293 shutdown_connections 2024-04-25T23:48:47.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.617+0000 7f0282df1640 1 -- 172.21.15.80:0/3109304293 wait complete. 2024-04-25T23:48:47.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.617+0000 7f0282df1640 1 Processor -- start 2024-04-25T23:48:47.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.617+0000 7f0282df1640 1 -- start start 2024-04-25T23:48:47.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.617+0000 7f0282df1640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f027c072060 0x7f027c133810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:47.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.617+0000 7f0282df1640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f027c0b1d80 0x7f027c1dc810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:47.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.617+0000 7f0282df1640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f027c13c840 0x7f027c1dcd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:47.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.617+0000 7f0282df1640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f027c14ef90 con 0x7f027c072060 2024-04-25T23:48:47.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.617+0000 7f0282df1640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f027c14ee10 con 0x7f027c0b1d80 2024-04-25T23:48:47.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.617+0000 7f0282df1640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f027c14f110 con 0x7f027c13c840 2024-04-25T23:48:47.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.621+0000 7f027bfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f027c0b1d80 0x7f027c1dc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:47.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.621+0000 7f027bfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f027c0b1d80 0x7f027c1dc810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:59120/0 (socket says 172.21.15.80:59120) 2024-04-25T23:48:47.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.621+0000 7f027bfff640 1 -- 172.21.15.80:0/2421793752 learned_addr learned my addr 172.21.15.80:0/2421793752 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T23:48:47.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.621+0000 7f028138e640 1 --2- 172.21.15.80:0/2421793752 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f027c13c840 0x7f027c1dcd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:47.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.621+0000 7f027bfff640 1 -- 172.21.15.80:0/2421793752 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f027c13c840 msgr2=0x7f027c1dcd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:47.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.621+0000 7f027bfff640 1 --2- 172.21.15.80:0/2421793752 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f027c13c840 0x7f027c1dcd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:47.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.621+0000 7f027bfff640 1 -- 172.21.15.80:0/2421793752 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f027c072060 msgr2=0x7f027c133810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:47.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.621+0000 7f0280b8d640 1 --2- 172.21.15.80:0/2421793752 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f027c072060 0x7f027c133810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:47.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.621+0000 7f027bfff640 1 --2- 172.21.15.80:0/2421793752 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f027c072060 0x7f027c133810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:47.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.621+0000 7f027bfff640 1 -- 172.21.15.80:0/2421793752 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f027c13ed40 con 0x7f027c0b1d80 2024-04-25T23:48:47.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.621+0000 7f027bfff640 1 --2- 172.21.15.80:0/2421793752 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f027c0b1d80 0x7f027c1dc810 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f026400c400 tx=0x7f026400c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:48:47.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.621+0000 7f02797fa640 1 -- 172.21.15.80:0/2421793752 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0264017070 con 0x7f027c0b1d80 2024-04-25T23:48:47.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.621+0000 7f02797fa640 1 -- 172.21.15.80:0/2421793752 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0264005cf0 con 0x7f027c0b1d80 2024-04-25T23:48:47.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.621+0000 7f0282df1640 1 -- 172.21.15.80:0/2421793752 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f027c13cd60 con 0x7f027c0b1d80 2024-04-25T23:48:47.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.621+0000 7f02797fa640 1 -- 172.21.15.80:0/2421793752 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02640072d0 con 0x7f027c0b1d80 2024-04-25T23:48:47.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.621+0000 7f0282df1640 1 -- 172.21.15.80:0/2421793752 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f027c1291f0 con 0x7f027c0b1d80 2024-04-25T23:48:47.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.621+0000 7f02797fa640 1 -- 172.21.15.80:0/2421793752 <== mon.1 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f026401a070 con 0x7f027c0b1d80 2024-04-25T23:48:47.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.621+0000 7f02797fa640 1 -- 172.21.15.80:0/2421793752 <== mon.1 v2:172.21.15.80:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f026401cc10 con 0x7f027c0b1d80 2024-04-25T23:48:47.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.621+0000 7f0282df1640 1 -- 172.21.15.80:0/2421793752 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f027c068210 con 0x7f027c0b1d80 2024-04-25T23:48:47.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.625+0000 7f02797fa640 1 -- 172.21.15.80:0/2421793752 <== mon.1 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f0264007470 con 0x7f027c0b1d80 2024-04-25T23:48:47.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.693+0000 7f0282df1640 1 -- 172.21.15.80:0/2421793752 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd new", "uuid": "1391be38-25ac-43f0-8090-15adddc6d69d", "id": 2} v 0) -- 0x7f027c13cf30 con 0x7f027c0b1d80 2024-04-25T23:48:47.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.705+0000 7f02797fa640 1 -- 172.21.15.80:0/2421793752 <== mon.1 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1391be38-25ac-43f0-8090-15adddc6d69d", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f026401ce10 con 0x7f027c0b1d80 2024-04-25T23:48:47.693 INFO:teuthology.orchestra.run.smithi080.stdout:2 2024-04-25T23:48:47.693 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-04-25T23:48:47.709+0000 7f625a234180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T23:48:47.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.709+0000 7f0282df1640 1 -- 172.21.15.80:0/2421793752 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f027c0b1d80 msgr2=0x7f027c1dc810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:47.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.709+0000 7f0282df1640 1 --2- 172.21.15.80:0/2421793752 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f027c0b1d80 0x7f027c1dc810 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f026400c400 tx=0x7f026400c8c0 comp rx=0 tx=0).stop 2024-04-25T23:48:47.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.709+0000 7f0282df1640 1 -- 172.21.15.80:0/2421793752 shutdown_connections 2024-04-25T23:48:47.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.709+0000 7f0282df1640 1 --2- 172.21.15.80:0/2421793752 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f027c0b1d80 0x7f027c1dc810 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:47.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.709+0000 7f0282df1640 1 --2- 172.21.15.80:0/2421793752 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f027c13c840 0x7f027c1dcd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:47.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.709+0000 7f0282df1640 1 --2- 172.21.15.80:0/2421793752 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f027c072060 0x7f027c133810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:47.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.709+0000 7f0282df1640 1 -- 172.21.15.80:0/2421793752 >> 172.21.15.80:0/2421793752 conn(0x7f027c0b2c10 msgr2=0x7f027c12d660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:48:47.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.709+0000 7f0282df1640 1 -- 172.21.15.80:0/2421793752 shutdown_connections 2024-04-25T23:48:47.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.709+0000 7f0282df1640 1 -- 172.21.15.80:0/2421793752 wait complete. 2024-04-25T23:48:47.710 DEBUG:teuthology.orchestra.run.smithi080:> sudo ceph --cluster ceph osd new a85e7650-effb-4ea7-9f3b-9fc9fc1576e4 3 2024-04-25T23:48:47.750 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-04-25T23:48:47.765+0000 7f625a234180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T23:48:47.801 INFO:tasks.ceph.mgr.y.smithi057.stderr:2024-04-25T23:48:47.815+0000 7f7104d7b180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T23:48:47.804 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-04-25T23:48:47.817+0000 7f625a234180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T23:48:47.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.833+0000 7fb251253640 1 Processor -- start 2024-04-25T23:48:47.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.833+0000 7fb251253640 1 -- start start 2024-04-25T23:48:47.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.837+0000 7fb251253640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fb24c13cc80 0x7fb24c13f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:47.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.837+0000 7fb251253640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb24c06f980 con 0x7fb24c071320 2024-04-25T23:48:47.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.837+0000 7fb251253640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb24c13ffc0 con 0x7fb24c070120 2024-04-25T23:48:47.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.837+0000 7fb251253640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7fb24c070500 con 0x7fb24c13cc80 2024-04-25T23:48:47.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.837+0000 7fb24ad9d640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fb24c070120 0x7fb24c13c570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:49044/0 (socket says 172.21.15.80:49044) 2024-04-25T23:48:47.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.837+0000 7fb24ad9d640 1 -- 172.21.15.80:0/2248962052 learned_addr learned my addr 172.21.15.80:0/2248962052 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T23:48:47.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.837+0000 7fb24a59c640 1 --2- 172.21.15.80:0/2248962052 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fb24c13cc80 0x7fb24c13f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:47.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.837+0000 7fb249d9b640 1 -- 172.21.15.80:0/2248962052 <== mon.1 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3712662289 0 0) 0x7fb24c13ffc0 con 0x7fb24c070120 2024-04-25T23:48:47.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.837+0000 7fb249d9b640 1 -- 172.21.15.80:0/2248962052 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb234003530 con 0x7fb24c070120 2024-04-25T23:48:47.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.837+0000 7fb249d9b640 1 -- 172.21.15.80:0/2248962052 <== mon.1 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 758903024 0 0) 0x7fb234003530 con 0x7fb24c070120 2024-04-25T23:48:47.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.837+0000 7fb249d9b640 1 -- 172.21.15.80:0/2248962052 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fb24c13cc80 msgr2=0x7fb24c13f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:48:47.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.837+0000 7fb249d9b640 1 --2- 172.21.15.80:0/2248962052 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fb24c13cc80 0x7fb24c13f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:47.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.837+0000 7fb249d9b640 1 -- 172.21.15.80:0/2248962052 >> v1:172.21.15.57:6789/0 conn(0x7fb24c071320 legacy=0x7fb24c13f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:47.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.837+0000 7fb249d9b640 1 -- 172.21.15.80:0/2248962052 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb24c073060 con 0x7fb24c070120 2024-04-25T23:48:47.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.837+0000 7fb249d9b640 1 -- 172.21.15.80:0/2248962052 <== mon.1 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2627952221 0 0) 0x7fb240001680 con 0x7fb24c070120 2024-04-25T23:48:47.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.837+0000 7fb249d9b640 1 -- 172.21.15.80:0/2248962052 >> v1:172.21.15.80:6789/0 conn(0x7fb24c070120 legacy=0x7fb24c13c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:47.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.837+0000 7fb249d9b640 1 --2- 172.21.15.80:0/2248962052 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb2340039c0 0x7fb234003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:47.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.837+0000 7fb249d9b640 1 --2- 172.21.15.80:0/2248962052 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fb234004810 0x7fb234004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:47.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.837+0000 7fb249d9b640 1 --2- 172.21.15.80:0/2248962052 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb234005190 0x7fb23400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:47.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.837+0000 7fb249d9b640 1 -- 172.21.15.80:0/2248962052 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fb24c070500 con 0x7fb234005190 2024-04-25T23:48:47.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.837+0000 7fb249d9b640 1 -- 172.21.15.80:0/2248962052 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb23400bf10 con 0x7fb2340039c0 2024-04-25T23:48:47.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.837+0000 7fb249d9b640 1 -- 172.21.15.80:0/2248962052 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7fb23400e2f0 con 0x7fb234004810 2024-04-25T23:48:47.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.837+0000 7fb24ad9d640 1 --2- 172.21.15.80:0/2248962052 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb2340039c0 0x7fb234003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:47.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.837+0000 7fb24a59c640 1 --2- 172.21.15.80:0/2248962052 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fb234004810 0x7fb234004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:47.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.837+0000 7fb24b59e640 1 --2- 172.21.15.80:0/2248962052 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb234005190 0x7fb23400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:47.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.837+0000 7fb24ad9d640 1 -- 172.21.15.80:0/2248962052 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fb234004810 msgr2=0x7fb234004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:48:47.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.837+0000 7fb24ad9d640 1 --2- 172.21.15.80:0/2248962052 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fb234004810 0x7fb234004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:47.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.837+0000 7fb24ad9d640 1 -- 172.21.15.80:0/2248962052 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb234005190 msgr2=0x7fb23400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:48:47.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.837+0000 7fb24ad9d640 1 --2- 172.21.15.80:0/2248962052 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb234005190 0x7fb23400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:47.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.837+0000 7fb24ad9d640 1 -- 172.21.15.80:0/2248962052 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb234003530 con 0x7fb2340039c0 2024-04-25T23:48:47.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.837+0000 7fb24b59e640 1 --2- 172.21.15.80:0/2248962052 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb234005190 0x7fb23400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T23:48:47.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.837+0000 7fb24a59c640 1 --2- 172.21.15.80:0/2248962052 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fb234004810 0x7fb234004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T23:48:47.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.837+0000 7fb24ad9d640 1 --2- 172.21.15.80:0/2248962052 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb2340039c0 0x7fb234003de0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb24000c270 tx=0x7fb24001f3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e52237d73e7ae651 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:48:47.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.837+0000 7fb249d9b640 1 -- 172.21.15.80:0/2248962052 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb240007d20 con 0x7fb2340039c0 2024-04-25T23:48:47.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.837+0000 7fb249d9b640 1 -- 172.21.15.80:0/2248962052 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb240005ae0 con 0x7fb2340039c0 2024-04-25T23:48:47.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.837+0000 7fb249d9b640 1 -- 172.21.15.80:0/2248962052 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb240007070 con 0x7fb2340039c0 2024-04-25T23:48:47.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.841+0000 7fb251253640 1 -- 172.21.15.80:0/2248962052 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb2340039c0 msgr2=0x7fb234003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:47.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.841+0000 7fb251253640 1 --2- 172.21.15.80:0/2248962052 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb2340039c0 0x7fb234003de0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb24000c270 tx=0x7fb24001f3b0 comp rx=0 tx=0).stop 2024-04-25T23:48:47.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.841+0000 7fb24b59e640 1 -- 172.21.15.80:0/2248962052 reap_dead start 2024-04-25T23:48:47.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.841+0000 7fb251253640 1 -- 172.21.15.80:0/2248962052 shutdown_connections 2024-04-25T23:48:47.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.841+0000 7fb251253640 1 -- 172.21.15.80:0/2248962052 >> 172.21.15.80:0/2248962052 conn(0x7fb24c0b51f0 msgr2=0x7fb24c0b5600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:48:47.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.841+0000 7fb251253640 1 -- 172.21.15.80:0/2248962052 shutdown_connections 2024-04-25T23:48:47.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.841+0000 7fb251253640 1 -- 172.21.15.80:0/2248962052 wait complete. 2024-04-25T23:48:47.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.841+0000 7fb251253640 1 Processor -- start 2024-04-25T23:48:47.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.841+0000 7fb251253640 1 -- start start 2024-04-25T23:48:47.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.841+0000 7fb251253640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2340039c0 0x7fb24c141b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:47.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.841+0000 7fb251253640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb24c142080 0x7fb24c1d7960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:47.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.841+0000 7fb251253640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fb24c140da0 0x7fb24c1d7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:47.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.841+0000 7fb251253640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fb24c13f4f0 con 0x7fb2340039c0 2024-04-25T23:48:47.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.841+0000 7fb251253640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb24c13f370 con 0x7fb24c142080 2024-04-25T23:48:47.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.841+0000 7fb251253640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7fb24c13f670 con 0x7fb24c140da0 2024-04-25T23:48:47.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.841+0000 7fb24a59c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb24c142080 0x7fb24c1d7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:47.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.841+0000 7fb24a59c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb24c142080 0x7fb24c1d7960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:59142/0 (socket says 172.21.15.80:59142) 2024-04-25T23:48:47.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.841+0000 7fb24a59c640 1 -- 172.21.15.80:0/3471677504 learned_addr learned my addr 172.21.15.80:0/3471677504 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T23:48:47.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.841+0000 7fb24ad9d640 1 --2- 172.21.15.80:0/3471677504 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2340039c0 0x7fb24c141b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:47.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.841+0000 7fb24b59e640 1 --2- 172.21.15.80:0/3471677504 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fb24c140da0 0x7fb24c1d7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:47.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.841+0000 7fb24a59c640 1 -- 172.21.15.80:0/3471677504 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fb24c140da0 msgr2=0x7fb24c1d7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:47.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.841+0000 7fb24a59c640 1 --2- 172.21.15.80:0/3471677504 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fb24c140da0 0x7fb24c1d7ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:47.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.841+0000 7fb24a59c640 1 -- 172.21.15.80:0/3471677504 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2340039c0 msgr2=0x7fb24c141b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:47.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.841+0000 7fb24a59c640 1 --2- 172.21.15.80:0/3471677504 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2340039c0 0x7fb24c141b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:47.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.841+0000 7fb24a59c640 1 -- 172.21.15.80:0/3471677504 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb24c0afd50 con 0x7fb24c142080 2024-04-25T23:48:47.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.841+0000 7fb24a59c640 1 --2- 172.21.15.80:0/3471677504 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb24c142080 0x7fb24c1d7960 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb23c010900 tx=0x7fb23c010dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:48:47.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.841+0000 7fb2337fe640 1 -- 172.21.15.80:0/3471677504 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb23c00f040 con 0x7fb24c142080 2024-04-25T23:48:47.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.841+0000 7fb2337fe640 1 -- 172.21.15.80:0/3471677504 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb23c00ec20 con 0x7fb24c142080 2024-04-25T23:48:47.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.841+0000 7fb251253640 1 -- 172.21.15.80:0/3471677504 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb24c0adf50 con 0x7fb24c142080 2024-04-25T23:48:47.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.841+0000 7fb2337fe640 1 -- 172.21.15.80:0/3471677504 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb23c01c400 con 0x7fb24c142080 2024-04-25T23:48:47.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.841+0000 7fb251253640 1 -- 172.21.15.80:0/3471677504 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb24c1daa70 con 0x7fb24c142080 2024-04-25T23:48:47.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.841+0000 7fb2337fe640 1 -- 172.21.15.80:0/3471677504 <== mon.1 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb23c0059a0 con 0x7fb24c142080 2024-04-25T23:48:47.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.841+0000 7fb2337fe640 1 -- 172.21.15.80:0/3471677504 <== mon.1 v2:172.21.15.80:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7fb23c0064b0 con 0x7fb24c142080 2024-04-25T23:48:47.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.841+0000 7fb251253640 1 -- 172.21.15.80:0/3471677504 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb24c068210 con 0x7fb24c142080 2024-04-25T23:48:47.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.845+0000 7fb2337fe640 1 -- 172.21.15.80:0/3471677504 <== mon.1 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fb23c02a080 con 0x7fb24c142080 2024-04-25T23:48:47.858 INFO:tasks.ceph.mgr.y.smithi057.stderr:2024-04-25T23:48:47.871+0000 7f7104d7b180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T23:48:47.883 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-04-25T23:48:47.897+0000 7f625a234180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T23:48:47.897 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.913+0000 7fb251253640 1 -- 172.21.15.80:0/3471677504 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a85e7650-effb-4ea7-9f3b-9fc9fc1576e4", "id": 3} v 0) -- 0x7fb24c06c7a0 con 0x7fb24c142080 2024-04-25T23:48:47.908 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.921+0000 7fb2337fe640 1 -- 172.21.15.80:0/3471677504 <== mon.1 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a85e7650-effb-4ea7-9f3b-9fc9fc1576e4", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7fb23c018030 con 0x7fb24c142080 2024-04-25T23:48:47.909 INFO:teuthology.orchestra.run.smithi080.stdout:3 2024-04-25T23:48:47.910 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.925+0000 7fb251253640 1 -- 172.21.15.80:0/3471677504 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb24c142080 msgr2=0x7fb24c1d7960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:47.910 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.925+0000 7fb251253640 1 --2- 172.21.15.80:0/3471677504 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb24c142080 0x7fb24c1d7960 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb23c010900 tx=0x7fb23c010dc0 comp rx=0 tx=0).stop 2024-04-25T23:48:47.910 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.925+0000 7fb251253640 1 -- 172.21.15.80:0/3471677504 shutdown_connections 2024-04-25T23:48:47.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.925+0000 7fb251253640 1 --2- 172.21.15.80:0/3471677504 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb24c142080 0x7fb24c1d7960 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:47.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.925+0000 7fb251253640 1 --2- 172.21.15.80:0/3471677504 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fb24c140da0 0x7fb24c1d7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:47.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.925+0000 7fb251253640 1 --2- 172.21.15.80:0/3471677504 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2340039c0 0x7fb24c141b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:47.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.925+0000 7fb251253640 1 -- 172.21.15.80:0/3471677504 >> 172.21.15.80:0/3471677504 conn(0x7fb24c0b51f0 msgr2=0x7fb24c0b1f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:48:47.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.925+0000 7fb251253640 1 -- 172.21.15.80:0/3471677504 shutdown_connections 2024-04-25T23:48:47.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:47.925+0000 7fb251253640 1 -- 172.21.15.80:0/3471677504 wait complete. 2024-04-25T23:48:47.914 INFO:tasks.ceph.mgr.y.smithi057.stderr:2024-04-25T23:48:47.927+0000 7f7104d7b180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T23:48:47.926 DEBUG:teuthology.orchestra.run.smithi080:> sudo ceph --cluster ceph osd new ad3fb1a9-cec6-4fb3-9f49-22b9480f524b 4 2024-04-25T23:48:47.938 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-04-25T23:48:47.953+0000 7f625a234180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T23:48:47.995 INFO:tasks.ceph.mgr.y.smithi057.stderr:2024-04-25T23:48:48.007+0000 7f7104d7b180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T23:48:48.018 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-04-25T23:48:48.033+0000 7f625a234180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T23:48:48.049 INFO:tasks.ceph.mgr.y.smithi057.stderr:2024-04-25T23:48:48.063+0000 7f7104d7b180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T23:48:48.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.065+0000 7f1a7bc1e640 1 Processor -- start 2024-04-25T23:48:48.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.065+0000 7f1a7bc1e640 1 -- start start 2024-04-25T23:48:48.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.069+0000 7f1a7bc1e640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1a74062720 0x7f1a74140ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:48.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.069+0000 7f1a7bc1e640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f1a7413d680 0x7f1a7413dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:48.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.069+0000 7f1a7bc1e640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a7413e120 con 0x7f1a74071280 2024-04-25T23:48:48.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.069+0000 7f1a7bc1e640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f1a74062b00 con 0x7f1a74062720 2024-04-25T23:48:48.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.069+0000 7f1a7bc1e640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f1a74141210 con 0x7f1a7413d680 2024-04-25T23:48:48.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.069+0000 7f1a799ba640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1a74062720 0x7f1a74140ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:48.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.069+0000 7f1a799ba640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1a74062720 0x7f1a74140ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:59150/0 (socket says 172.21.15.80:59150) 2024-04-25T23:48:48.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.069+0000 7f1a799ba640 1 -- 172.21.15.80:0/2455565232 learned_addr learned my addr 172.21.15.80:0/2455565232 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T23:48:48.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.069+0000 7f1a791b9640 1 --2- 172.21.15.80:0/2455565232 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f1a7413d680 0x7f1a7413dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:48.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.069+0000 7f1a799ba640 1 -- 172.21.15.80:0/2455565232 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f1a7413d680 msgr2=0x7f1a7413dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:48:48.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.069+0000 7f1a799ba640 1 --2- 172.21.15.80:0/2455565232 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f1a7413d680 0x7f1a7413dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:48.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.069+0000 7f1a799ba640 1 -- 172.21.15.80:0/2455565232 >> v1:172.21.15.57:6789/0 conn(0x7f1a74071280 legacy=0x7f1a740716f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:48:48.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.069+0000 7f1a799ba640 1 -- 172.21.15.80:0/2455565232 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a740b10a0 con 0x7f1a74062720 2024-04-25T23:48:48.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.069+0000 7f1a791b9640 1 --2- 172.21.15.80:0/2455565232 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f1a7413d680 0x7f1a7413dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T23:48:48.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.069+0000 7f1a799ba640 1 --2- 172.21.15.80:0/2455565232 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a74062720 0x7f1a74140ac0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1a70009870 tx=0x7f1a7001ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8d0a6e391b9a45 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:48:48.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.069+0000 7f1a789b8640 1 -- 172.21.15.80:0/2455565232 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a700b5070 con 0x7f1a74062720 2024-04-25T23:48:48.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.069+0000 7f1a789b8640 1 -- 172.21.15.80:0/2455565232 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a700a8a30 con 0x7f1a74062720 2024-04-25T23:48:48.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.069+0000 7f1a789b8640 1 -- 172.21.15.80:0/2455565232 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a700a8e00 con 0x7f1a74062720 2024-04-25T23:48:48.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.069+0000 7f1a7bc1e640 1 -- 172.21.15.80:0/2455565232 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a74062720 msgr2=0x7f1a74140ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:48.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.069+0000 7f1a7bc1e640 1 --2- 172.21.15.80:0/2455565232 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a74062720 0x7f1a74140ac0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1a70009870 tx=0x7f1a7001ed30 comp rx=0 tx=0).stop 2024-04-25T23:48:48.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.069+0000 7f1a7bc1e640 1 -- 172.21.15.80:0/2455565232 shutdown_connections 2024-04-25T23:48:48.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.069+0000 7f1a7bc1e640 1 --2- 172.21.15.80:0/2455565232 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f1a7413d680 0x7f1a7413dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:48.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.069+0000 7f1a7bc1e640 1 --2- 172.21.15.80:0/2455565232 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a74062720 0x7f1a74140ac0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:48.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.069+0000 7f1a7bc1e640 1 -- 172.21.15.80:0/2455565232 >> 172.21.15.80:0/2455565232 conn(0x7f1a7406b810 msgr2=0x7f1a740b4890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:48:48.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.069+0000 7f1a7bc1e640 1 -- 172.21.15.80:0/2455565232 shutdown_connections 2024-04-25T23:48:48.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.069+0000 7f1a7bc1e640 1 -- 172.21.15.80:0/2455565232 wait complete. 2024-04-25T23:48:48.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.073+0000 7f1a7bc1e640 1 Processor -- start 2024-04-25T23:48:48.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.073+0000 7f1a7bc1e640 1 -- start start 2024-04-25T23:48:48.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.073+0000 7f1a7bc1e640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a74062720 0x7f1a74138ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:48.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.073+0000 7f1a7bc1e640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f1a74071280 0x7f1a741dbd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:48.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.073+0000 7f1a7bc1e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a7413d680 0x7f1a7406f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:48.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.073+0000 7f1a7bc1e640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f1a7414b070 con 0x7f1a74062720 2024-04-25T23:48:48.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.073+0000 7f1a7bc1e640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f1a7414aef0 con 0x7f1a7413d680 2024-04-25T23:48:48.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.073+0000 7f1a7bc1e640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f1a7414b1f0 con 0x7f1a74071280 2024-04-25T23:48:48.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.073+0000 7f1a7a1bb640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a7413d680 0x7f1a7406f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:48.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.073+0000 7f1a7a1bb640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a7413d680 0x7f1a7406f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:59156/0 (socket says 172.21.15.80:59156) 2024-04-25T23:48:48.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.073+0000 7f1a7a1bb640 1 -- 172.21.15.80:0/4261945384 learned_addr learned my addr 172.21.15.80:0/4261945384 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T23:48:48.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.073+0000 7f1a799ba640 1 --2- 172.21.15.80:0/4261945384 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a74062720 0x7f1a74138ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:48.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.073+0000 7f1a791b9640 1 --2- 172.21.15.80:0/4261945384 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f1a74071280 0x7f1a741dbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:48.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.073+0000 7f1a7a1bb640 1 -- 172.21.15.80:0/4261945384 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f1a74071280 msgr2=0x7f1a741dbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:48.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.073+0000 7f1a7a1bb640 1 --2- 172.21.15.80:0/4261945384 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f1a74071280 0x7f1a741dbd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:48.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.073+0000 7f1a7a1bb640 1 -- 172.21.15.80:0/4261945384 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a74062720 msgr2=0x7f1a74138ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:48.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.073+0000 7f1a7a1bb640 1 --2- 172.21.15.80:0/4261945384 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a74062720 0x7f1a74138ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:48.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.073+0000 7f1a7a1bb640 1 -- 172.21.15.80:0/4261945384 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a741dc2d0 con 0x7f1a7413d680 2024-04-25T23:48:48.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.073+0000 7f1a7a1bb640 1 --2- 172.21.15.80:0/4261945384 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a7413d680 0x7f1a7406f070 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c00c7d0 tx=0x7f1a6c00cc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:48:48.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.073+0000 7f1a6a7fc640 1 -- 172.21.15.80:0/4261945384 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a6c003b80 con 0x7f1a7413d680 2024-04-25T23:48:48.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.073+0000 7f1a6a7fc640 1 -- 172.21.15.80:0/4261945384 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a6c003d20 con 0x7f1a7413d680 2024-04-25T23:48:48.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.073+0000 7f1a7bc1e640 1 -- 172.21.15.80:0/4261945384 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a74070ed0 con 0x7f1a7413d680 2024-04-25T23:48:48.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.073+0000 7f1a6a7fc640 1 -- 172.21.15.80:0/4261945384 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a6c00aa90 con 0x7f1a7413d680 2024-04-25T23:48:48.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.073+0000 7f1a7bc1e640 1 -- 172.21.15.80:0/4261945384 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a7406f5b0 con 0x7f1a7413d680 2024-04-25T23:48:48.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.073+0000 7f1a6a7fc640 1 -- 172.21.15.80:0/4261945384 <== mon.1 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1a6c00ac30 con 0x7f1a7413d680 2024-04-25T23:48:48.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.073+0000 7f1a6a7fc640 1 -- 172.21.15.80:0/4261945384 <== mon.1 v2:172.21.15.80:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f1a6c012ac0 con 0x7f1a7413d680 2024-04-25T23:48:48.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.073+0000 7f1a7bc1e640 1 -- 172.21.15.80:0/4261945384 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a74068210 con 0x7f1a7413d680 2024-04-25T23:48:48.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.077+0000 7f1a6a7fc640 1 -- 172.21.15.80:0/4261945384 <== mon.1 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f1a6c012d10 con 0x7f1a7413d680 2024-04-25T23:48:48.073 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-04-25T23:48:48.089+0000 7f625a234180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T23:48:48.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.145+0000 7f1a7bc1e640 1 -- 172.21.15.80:0/4261945384 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ad3fb1a9-cec6-4fb3-9f49-22b9480f524b", "id": 4} v 0) -- 0x7f1a7414acd0 con 0x7f1a7413d680 2024-04-25T23:48:48.131 INFO:tasks.ceph.mgr.y.smithi057.stderr:2024-04-25T23:48:48.143+0000 7f7104d7b180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T23:48:48.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.161+0000 7f1a6a7fc640 1 -- 172.21.15.80:0/4261945384 <== mon.1 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ad3fb1a9-cec6-4fb3-9f49-22b9480f524b", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f1a6c0125b0 con 0x7f1a7413d680 2024-04-25T23:48:48.145 INFO:teuthology.orchestra.run.smithi080.stdout:4 2024-04-25T23:48:48.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.161+0000 7f1a7bc1e640 1 -- 172.21.15.80:0/4261945384 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a7413d680 msgr2=0x7f1a7406f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:48.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.161+0000 7f1a7bc1e640 1 --2- 172.21.15.80:0/4261945384 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a7413d680 0x7f1a7406f070 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c00c7d0 tx=0x7f1a6c00cc90 comp rx=0 tx=0).stop 2024-04-25T23:48:48.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.161+0000 7f1a7bc1e640 1 -- 172.21.15.80:0/4261945384 shutdown_connections 2024-04-25T23:48:48.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.161+0000 7f1a7bc1e640 1 --2- 172.21.15.80:0/4261945384 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a7413d680 0x7f1a7406f070 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:48.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.161+0000 7f1a7bc1e640 1 --2- 172.21.15.80:0/4261945384 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f1a74071280 0x7f1a741dbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:48.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.161+0000 7f1a7bc1e640 1 --2- 172.21.15.80:0/4261945384 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1a74062720 0x7f1a74138ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:48.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.161+0000 7f1a7bc1e640 1 -- 172.21.15.80:0/4261945384 >> 172.21.15.80:0/4261945384 conn(0x7f1a7406b810 msgr2=0x7f1a7413bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:48:48.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.161+0000 7f1a7bc1e640 1 -- 172.21.15.80:0/4261945384 shutdown_connections 2024-04-25T23:48:48.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.161+0000 7f1a7bc1e640 1 -- 172.21.15.80:0/4261945384 wait complete. 2024-04-25T23:48:48.161 DEBUG:teuthology.orchestra.run.smithi080:> sudo ceph --cluster ceph osd new fd05cda6-27a5-477a-8916-b34b69ba9ef6 5 2024-04-25T23:48:48.188 INFO:tasks.ceph.mgr.y.smithi057.stderr:2024-04-25T23:48:48.199+0000 7f7104d7b180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T23:48:48.231 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-04-25T23:48:48.245+0000 7f625a234180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T23:48:48.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.285+0000 7fdcf6226640 1 Processor -- start 2024-04-25T23:48:48.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.285+0000 7fdcf6226640 1 -- start start 2024-04-25T23:48:48.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.289+0000 7fdcf6226640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fdcf00b1d80 0x7fdcf0072060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:48.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.289+0000 7fdcf6226640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fdcf00b2730 0x7fdcf013a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:48.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.289+0000 7fdcf6226640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcf00a95e0 con 0x7fdcf013a620 2024-04-25T23:48:48.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.289+0000 7fdcf6226640 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 -- 0x7fdcf00b0890 con 0x7fdcf00b1d80 2024-04-25T23:48:48.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.289+0000 7fdcf6226640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fdcf00b2160 con 0x7fdcf00b2730 2024-04-25T23:48:48.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.289+0000 7fdceeffd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fdcf00b2730 0x7fdcf013a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:48.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.289+0000 7fdceeffd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fdcf00b2730 0x7fdcf013a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:59164/0 (socket says 172.21.15.80:59164) 2024-04-25T23:48:48.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.289+0000 7fdceeffd640 1 -- 172.21.15.80:0/913693466 learned_addr learned my addr 172.21.15.80:0/913693466 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T23:48:48.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.289+0000 7fdcef7fe640 1 --2- 172.21.15.80:0/913693466 >> v2:172.21.15.57:3300/0 conn(0x7fdcf00b1d80 0x7fdcf0072060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:48.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.289+0000 7fdceeffd640 1 -- 172.21.15.80:0/913693466 >> v2:172.21.15.57:3300/0 conn(0x7fdcf00b1d80 msgr2=0x7fdcf0072060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:48:48.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.289+0000 7fdceeffd640 1 --2- 172.21.15.80:0/913693466 >> v2:172.21.15.57:3300/0 conn(0x7fdcf00b1d80 0x7fdcf0072060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:48.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.289+0000 7fdceeffd640 1 -- 172.21.15.80:0/913693466 >> v1:172.21.15.57:6789/0 conn(0x7fdcf013a620 legacy=0x7fdcf0143b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:48.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.289+0000 7fdceeffd640 1 -- 172.21.15.80:0/913693466 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcf00726d0 con 0x7fdcf00b2730 2024-04-25T23:48:48.276 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.289+0000 7fdceeffd640 1 --2- 172.21.15.80:0/913693466 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdcf00b2730 0x7fdcf013a0e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fdcf00b38c0 tx=0x7fdce001c500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6730c403c868259a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:48:48.276 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.289+0000 7fdcee7fc640 1 -- 172.21.15.80:0/913693466 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce00aa680 con 0x7fdcf00b2730 2024-04-25T23:48:48.276 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.289+0000 7fdcee7fc640 1 -- 172.21.15.80:0/913693466 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdce00af070 con 0x7fdcf00b2730 2024-04-25T23:48:48.276 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.289+0000 7fdcee7fc640 1 -- 172.21.15.80:0/913693466 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce00aaa40 con 0x7fdcf00b2730 2024-04-25T23:48:48.277 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.289+0000 7fdcf6226640 1 -- 172.21.15.80:0/913693466 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdcf00b2730 msgr2=0x7fdcf013a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:48.277 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.289+0000 7fdcf6226640 1 --2- 172.21.15.80:0/913693466 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdcf00b2730 0x7fdcf013a0e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fdcf00b38c0 tx=0x7fdce001c500 comp rx=0 tx=0).stop 2024-04-25T23:48:48.277 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.289+0000 7fdcf6226640 1 -- 172.21.15.80:0/913693466 shutdown_connections 2024-04-25T23:48:48.277 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.289+0000 7fdcf6226640 1 --2- 172.21.15.80:0/913693466 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdcf00b2730 0x7fdcf013a0e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:48.277 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.289+0000 7fdcf6226640 1 --2- 172.21.15.80:0/913693466 >> v2:172.21.15.57:3300/0 conn(0x7fdcf00b1d80 0x7fdcf0072060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:48.277 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.289+0000 7fdcf6226640 1 -- 172.21.15.80:0/913693466 >> 172.21.15.80:0/913693466 conn(0x7fdcf00b2c10 msgr2=0x7fdcf00b3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:48:48.277 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.289+0000 7fdcf6226640 1 -- 172.21.15.80:0/913693466 shutdown_connections 2024-04-25T23:48:48.277 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.289+0000 7fdcf6226640 1 -- 172.21.15.80:0/913693466 wait complete. 2024-04-25T23:48:48.278 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.293+0000 7fdcf6226640 1 Processor -- start 2024-04-25T23:48:48.278 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.293+0000 7fdcf6226640 1 -- start start 2024-04-25T23:48:48.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.293+0000 7fdcf6226640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdcf00b1d80 0x7fdcf0140d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:48.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.293+0000 7fdcf6226640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fdcf013a620 0x7fdcf013e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:48.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.293+0000 7fdcf6226640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdcf013ee00 0x7fdcf01dbd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:48.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.293+0000 7fdcf6226640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf014d940 con 0x7fdcf00b1d80 2024-04-25T23:48:48.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.293+0000 7fdcf6226640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf014d7c0 con 0x7fdcf013ee00 2024-04-25T23:48:48.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.293+0000 7fdcf6226640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7fdcf014dac0 con 0x7fdcf013a620 2024-04-25T23:48:48.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.293+0000 7fdceffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdcf013ee00 0x7fdcf01dbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:48.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.293+0000 7fdceffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdcf013ee00 0x7fdcf01dbd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:59178/0 (socket says 172.21.15.80:59178) 2024-04-25T23:48:48.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.293+0000 7fdceffff640 1 -- 172.21.15.80:0/2094242091 learned_addr learned my addr 172.21.15.80:0/2094242091 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T23:48:48.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.293+0000 7fdcef7fe640 1 --2- 172.21.15.80:0/2094242091 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdcf00b1d80 0x7fdcf0140d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:48.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.293+0000 7fdceeffd640 1 --2- 172.21.15.80:0/2094242091 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fdcf013a620 0x7fdcf013e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:48.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.293+0000 7fdceffff640 1 -- 172.21.15.80:0/2094242091 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fdcf013a620 msgr2=0x7fdcf013e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:48.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.293+0000 7fdceffff640 1 --2- 172.21.15.80:0/2094242091 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fdcf013a620 0x7fdcf013e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:48.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.293+0000 7fdceffff640 1 -- 172.21.15.80:0/2094242091 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdcf00b1d80 msgr2=0x7fdcf0140d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:48.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.293+0000 7fdceffff640 1 --2- 172.21.15.80:0/2094242091 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdcf00b1d80 0x7fdcf0140d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:48.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.293+0000 7fdceffff640 1 -- 172.21.15.80:0/2094242091 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcf013fd60 con 0x7fdcf013ee00 2024-04-25T23:48:48.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.293+0000 7fdceffff640 1 --2- 172.21.15.80:0/2094242091 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdcf013ee00 0x7fdcf01dbd90 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fdce800c730 tx=0x7fdce800cbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:48:48.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.293+0000 7fdcd7fff640 1 -- 172.21.15.80:0/2094242091 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce8003b80 con 0x7fdcf013ee00 2024-04-25T23:48:48.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.293+0000 7fdcd7fff640 1 -- 172.21.15.80:0/2094242091 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdce8003d20 con 0x7fdcf013ee00 2024-04-25T23:48:48.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.293+0000 7fdcf6226640 1 -- 172.21.15.80:0/2094242091 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcf0140040 con 0x7fdcf013ee00 2024-04-25T23:48:48.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.293+0000 7fdcd7fff640 1 -- 172.21.15.80:0/2094242091 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce8005310 con 0x7fdcf013ee00 2024-04-25T23:48:48.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.293+0000 7fdcf6226640 1 -- 172.21.15.80:0/2094242091 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcf0138060 con 0x7fdcf013ee00 2024-04-25T23:48:48.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.293+0000 7fdcd7fff640 1 -- 172.21.15.80:0/2094242091 <== mon.1 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdce80056c0 con 0x7fdcf013ee00 2024-04-25T23:48:48.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.293+0000 7fdcd7fff640 1 -- 172.21.15.80:0/2094242091 <== mon.1 v2:172.21.15.80:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7fdce8012c70 con 0x7fdcf013ee00 2024-04-25T23:48:48.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.293+0000 7fdcf6226640 1 -- 172.21.15.80:0/2094242091 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcf0068210 con 0x7fdcf013ee00 2024-04-25T23:48:48.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.297+0000 7fdcd7fff640 1 -- 172.21.15.80:0/2094242091 <== mon.1 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fdce80126f0 con 0x7fdcf013ee00 2024-04-25T23:48:48.293 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-04-25T23:48:48.309+0000 7f625a234180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T23:48:48.346 INFO:tasks.ceph.mgr.y.smithi057.stderr:2024-04-25T23:48:48.359+0000 7f7104d7b180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T23:48:48.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.365+0000 7fdcf6226640 1 -- 172.21.15.80:0/2094242091 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd new", "uuid": "fd05cda6-27a5-477a-8916-b34b69ba9ef6", "id": 5} v 0) -- 0x7fdcf01403d0 con 0x7fdcf013ee00 2024-04-25T23:48:48.359 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.373+0000 7fdcd7fff640 1 -- 172.21.15.80:0/2094242091 <== mon.1 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "fd05cda6-27a5-477a-8916-b34b69ba9ef6", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7fdce80128d0 con 0x7fdcf013ee00 2024-04-25T23:48:48.359 INFO:teuthology.orchestra.run.smithi080.stdout:5 2024-04-25T23:48:48.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.373+0000 7fdcf6226640 1 -- 172.21.15.80:0/2094242091 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdcf013ee00 msgr2=0x7fdcf01dbd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:48.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.373+0000 7fdcf6226640 1 --2- 172.21.15.80:0/2094242091 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdcf013ee00 0x7fdcf01dbd90 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fdce800c730 tx=0x7fdce800cbf0 comp rx=0 tx=0).stop 2024-04-25T23:48:48.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.373+0000 7fdcf6226640 1 -- 172.21.15.80:0/2094242091 shutdown_connections 2024-04-25T23:48:48.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.373+0000 7fdcf6226640 1 --2- 172.21.15.80:0/2094242091 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdcf013ee00 0x7fdcf01dbd90 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:48.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.373+0000 7fdcf6226640 1 --2- 172.21.15.80:0/2094242091 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fdcf013a620 0x7fdcf013e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:48.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.373+0000 7fdcf6226640 1 --2- 172.21.15.80:0/2094242091 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdcf00b1d80 0x7fdcf0140d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:48.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.373+0000 7fdcf6226640 1 -- 172.21.15.80:0/2094242091 >> 172.21.15.80:0/2094242091 conn(0x7fdcf00b2c10 msgr2=0x7fdcf00b2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:48:48.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.373+0000 7fdcf6226640 1 -- 172.21.15.80:0/2094242091 shutdown_connections 2024-04-25T23:48:48.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.373+0000 7fdcf6226640 1 -- 172.21.15.80:0/2094242091 wait complete. 2024-04-25T23:48:48.376 DEBUG:teuthology.orchestra.run.smithi080:> sudo ceph --cluster ceph osd new 599ef421-483d-4225-a98f-7405006b1809 6 2024-04-25T23:48:48.411 INFO:tasks.ceph.mgr.y.smithi057.stderr:2024-04-25T23:48:48.423+0000 7f7104d7b180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T23:48:48.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.501+0000 7f5ea9625640 1 Processor -- start 2024-04-25T23:48:48.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.501+0000 7f5ea9625640 1 -- start start 2024-04-25T23:48:48.488 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.501+0000 7f5ea9625640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5ea40b0140 0x7f5ea4140690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:48.489 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.501+0000 7f5ea9625640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ea40b0940 con 0x7f5ea4140bd0 2024-04-25T23:48:48.489 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.501+0000 7f5ea9625640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ea413a710 con 0x7f5ea4074b50 2024-04-25T23:48:48.489 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.501+0000 7f5ea9625640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f5ea4074f30 con 0x7f5ea40b0140 2024-04-25T23:48:48.489 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.501+0000 7f5e9bfff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5ea40b0140 0x7f5ea4140690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:48.489 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.501+0000 7f5e9bfff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5ea40b0140 0x7f5ea4140690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:59194/0 (socket says 172.21.15.80:59194) 2024-04-25T23:48:48.489 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.501+0000 7f5e9bfff640 1 -- 172.21.15.80:0/2147274331 learned_addr learned my addr 172.21.15.80:0/2147274331 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T23:48:48.489 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.501+0000 7f5e9bfff640 1 -- 172.21.15.80:0/2147274331 >> v1:172.21.15.80:6789/0 conn(0x7f5ea4074b50 legacy=0x7f5ea413a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:48.489 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.501+0000 7f5e9bfff640 1 -- 172.21.15.80:0/2147274331 >> v1:172.21.15.57:6789/0 conn(0x7f5ea4140bd0 legacy=0x7f5ea414acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:48:48.489 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.501+0000 7f5e9bfff640 1 -- 172.21.15.80:0/2147274331 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ea4073420 con 0x7f5ea40b0140 2024-04-25T23:48:48.490 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.505+0000 7f5e9bfff640 1 --2- 172.21.15.80:0/2147274331 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ea40b0140 0x7f5ea4140690 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5e94009870 tx=0x7f5e9401c500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cad7f7832618f56c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:48:48.491 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.505+0000 7f5ea27fc640 1 -- 172.21.15.80:0/2147274331 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e940b0070 con 0x7f5ea40b0140 2024-04-25T23:48:48.491 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.505+0000 7f5ea27fc640 1 -- 172.21.15.80:0/2147274331 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e940ab600 con 0x7f5ea40b0140 2024-04-25T23:48:48.491 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.505+0000 7f5ea27fc640 1 -- 172.21.15.80:0/2147274331 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e940ab9d0 con 0x7f5ea40b0140 2024-04-25T23:48:48.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.505+0000 7f5ea9625640 1 -- 172.21.15.80:0/2147274331 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ea40b0140 msgr2=0x7f5ea4140690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:48.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.505+0000 7f5ea9625640 1 --2- 172.21.15.80:0/2147274331 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ea40b0140 0x7f5ea4140690 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5e94009870 tx=0x7f5e9401c500 comp rx=0 tx=0).stop 2024-04-25T23:48:48.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.505+0000 7f5ea9625640 1 -- 172.21.15.80:0/2147274331 shutdown_connections 2024-04-25T23:48:48.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.505+0000 7f5ea9625640 1 --2- 172.21.15.80:0/2147274331 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ea40b0140 0x7f5ea4140690 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:48.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.505+0000 7f5ea9625640 1 -- 172.21.15.80:0/2147274331 >> 172.21.15.80:0/2147274331 conn(0x7f5ea40b51f0 msgr2=0x7f5ea40b5600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:48:48.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.505+0000 7f5ea9625640 1 -- 172.21.15.80:0/2147274331 shutdown_connections 2024-04-25T23:48:48.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.505+0000 7f5ea9625640 1 -- 172.21.15.80:0/2147274331 wait complete. 2024-04-25T23:48:48.494 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.505+0000 7f5ea9625640 1 Processor -- start 2024-04-25T23:48:48.494 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.505+0000 7f5ea9625640 1 -- start start 2024-04-25T23:48:48.494 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.509+0000 7f5ea9625640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f5ea4074b50 0x7f5ea40707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:48.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.509+0000 7f5ea9625640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ea40b0140 0x7f5ea41416f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:48.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.509+0000 7f5ea9625640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ea4140bd0 0x7f5ea4142da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:48.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.509+0000 7f5ea9625640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea4141250 con 0x7f5ea4140bd0 2024-04-25T23:48:48.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.509+0000 7f5ea9625640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea40af730 con 0x7f5ea40b0140 2024-04-25T23:48:48.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.509+0000 7f5ea9625640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f5ea414ee10 con 0x7f5ea4074b50 2024-04-25T23:48:48.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.509+0000 7f5e9bfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ea40b0140 0x7f5ea41416f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:48.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.509+0000 7f5e9bfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ea40b0140 0x7f5ea41416f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:59200/0 (socket says 172.21.15.80:59200) 2024-04-25T23:48:48.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.509+0000 7f5e9bfff640 1 -- 172.21.15.80:0/3112913837 learned_addr learned my addr 172.21.15.80:0/3112913837 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T23:48:48.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.509+0000 7f5ea2ffd640 1 --2- 172.21.15.80:0/3112913837 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f5ea4074b50 0x7f5ea40707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:48.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.509+0000 7f5ea37fe640 1 --2- 172.21.15.80:0/3112913837 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ea4140bd0 0x7f5ea4142da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:48.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.509+0000 7f5e9bfff640 1 -- 172.21.15.80:0/3112913837 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f5ea4074b50 msgr2=0x7f5ea40707f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:48.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.509+0000 7f5e9bfff640 1 --2- 172.21.15.80:0/3112913837 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f5ea4074b50 0x7f5ea40707f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:48.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.509+0000 7f5e9bfff640 1 -- 172.21.15.80:0/3112913837 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ea4140bd0 msgr2=0x7f5ea4142da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:48.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.509+0000 7f5e9bfff640 1 --2- 172.21.15.80:0/3112913837 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ea4140bd0 0x7f5ea4142da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:48.496 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.509+0000 7f5e9bfff640 1 -- 172.21.15.80:0/3112913837 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ea41432e0 con 0x7f5ea40b0140 2024-04-25T23:48:48.496 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.509+0000 7f5ea37fe640 1 --2- 172.21.15.80:0/3112913837 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ea4140bd0 0x7f5ea4142da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T23:48:48.496 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.509+0000 7f5e9bfff640 1 --2- 172.21.15.80:0/3112913837 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ea40b0140 0x7f5ea41416f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5e940ae040 tx=0x7f5e940192a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:48:48.496 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.509+0000 7f5e9b7fe640 1 -- 172.21.15.80:0/3112913837 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e940b0040 con 0x7f5ea40b0140 2024-04-25T23:48:48.496 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.509+0000 7f5e9b7fe640 1 -- 172.21.15.80:0/3112913837 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e940b9070 con 0x7f5ea40b0140 2024-04-25T23:48:48.496 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.509+0000 7f5ea9625640 1 -- 172.21.15.80:0/3112913837 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ea413b3f0 con 0x7f5ea40b0140 2024-04-25T23:48:48.496 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.509+0000 7f5e9b7fe640 1 -- 172.21.15.80:0/3112913837 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e940b4860 con 0x7f5ea40b0140 2024-04-25T23:48:48.496 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.509+0000 7f5ea9625640 1 -- 172.21.15.80:0/3112913837 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ea4143c40 con 0x7f5ea40b0140 2024-04-25T23:48:48.496 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.509+0000 7f5e9b7fe640 1 -- 172.21.15.80:0/3112913837 <== mon.1 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5e940ab4c0 con 0x7f5ea40b0140 2024-04-25T23:48:48.496 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.509+0000 7f5e9b7fe640 1 -- 172.21.15.80:0/3112913837 <== mon.1 v2:172.21.15.80:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f5e940bd3c0 con 0x7f5ea40b0140 2024-04-25T23:48:48.496 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.509+0000 7f5ea9625640 1 -- 172.21.15.80:0/3112913837 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ea413b210 con 0x7f5ea40b0140 2024-04-25T23:48:48.497 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.509+0000 7f5e9b7fe640 1 -- 172.21.15.80:0/3112913837 <== mon.1 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f5ea413b210 con 0x7f5ea40b0140 2024-04-25T23:48:48.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.577+0000 7f5ea9625640 1 -- 172.21.15.80:0/3112913837 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd new", "uuid": "599ef421-483d-4225-a98f-7405006b1809", "id": 6} v 0) -- 0x7f5ea4068210 con 0x7f5ea40b0140 2024-04-25T23:48:48.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.585+0000 7f5e9b7fe640 1 -- 172.21.15.80:0/3112913837 <== mon.1 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "599ef421-483d-4225-a98f-7405006b1809", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f5e940160f0 con 0x7f5ea40b0140 2024-04-25T23:48:48.573 INFO:teuthology.orchestra.run.smithi080.stdout:6 2024-04-25T23:48:48.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.589+0000 7f5ea9625640 1 -- 172.21.15.80:0/3112913837 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ea40b0140 msgr2=0x7f5ea41416f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:48.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.589+0000 7f5ea9625640 1 --2- 172.21.15.80:0/3112913837 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ea40b0140 0x7f5ea41416f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5e940ae040 tx=0x7f5e940192a0 comp rx=0 tx=0).stop 2024-04-25T23:48:48.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.589+0000 7f5ea9625640 1 -- 172.21.15.80:0/3112913837 shutdown_connections 2024-04-25T23:48:48.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.589+0000 7f5ea9625640 1 --2- 172.21.15.80:0/3112913837 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ea40b0140 0x7f5ea41416f0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:48.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.589+0000 7f5ea9625640 1 --2- 172.21.15.80:0/3112913837 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ea4140bd0 0x7f5ea4142da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:48.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.589+0000 7f5ea9625640 1 --2- 172.21.15.80:0/3112913837 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f5ea4074b50 0x7f5ea40707f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:48.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.589+0000 7f5ea9625640 1 -- 172.21.15.80:0/3112913837 >> 172.21.15.80:0/3112913837 conn(0x7f5ea40b51f0 msgr2=0x7f5ea40b3a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:48:48.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.589+0000 7f5ea9625640 1 -- 172.21.15.80:0/3112913837 shutdown_connections 2024-04-25T23:48:48.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.589+0000 7f5ea9625640 1 -- 172.21.15.80:0/3112913837 wait complete. 2024-04-25T23:48:48.589 DEBUG:teuthology.orchestra.run.smithi080:> sudo ceph --cluster ceph osd new 0e9d533c-465f-403c-8fe8-7dafc953d774 7 2024-04-25T23:48:48.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.749+0000 7f023d1e8640 1 Processor -- start 2024-04-25T23:48:48.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.749+0000 7f023d1e8640 1 -- start start 2024-04-25T23:48:48.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.749+0000 7f023d1e8640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f02380b54f0 0x7f0238146010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:48.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.749+0000 7f023d1e8640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f02380b2530 0x7f023814cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:48.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.749+0000 7f023d1e8640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02380b2960 con 0x7f0238146550 2024-04-25T23:48:48.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.749+0000 7f023d1e8640 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 -- 0x7f02380b58d0 con 0x7f02380b54f0 2024-04-25T23:48:48.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.749+0000 7f023d1e8640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f0238145b80 con 0x7f02380b2530 2024-04-25T23:48:48.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.749+0000 7f023759e640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f02380b2530 0x7f023814cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:48.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.749+0000 7f023759e640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f02380b2530 0x7f023814cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:59216/0 (socket says 172.21.15.80:59216) 2024-04-25T23:48:48.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.749+0000 7f023759e640 1 -- 172.21.15.80:0/4197335776 learned_addr learned my addr 172.21.15.80:0/4197335776 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T23:48:48.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.749+0000 7f0236d9d640 1 --2- 172.21.15.80:0/4197335776 >> v2:172.21.15.57:3300/0 conn(0x7f02380b54f0 0x7f0238146010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:48.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.749+0000 7f023759e640 1 -- 172.21.15.80:0/4197335776 >> v2:172.21.15.57:3300/0 conn(0x7f02380b54f0 msgr2=0x7f0238146010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:48:48.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.749+0000 7f023759e640 1 --2- 172.21.15.80:0/4197335776 >> v2:172.21.15.57:3300/0 conn(0x7f02380b54f0 0x7f0238146010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:48.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.749+0000 7f023759e640 1 -- 172.21.15.80:0/4197335776 >> v1:172.21.15.80:6789/0 conn(0x7f0238146550 legacy=0x7f02380b1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:48.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.749+0000 7f023759e640 1 -- 172.21.15.80:0/4197335776 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0238142600 con 0x7f02380b2530 2024-04-25T23:48:48.738 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.753+0000 7f023759e640 1 --2- 172.21.15.80:0/4197335776 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02380b2530 0x7f023814cf10 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f02380aa230 tx=0x7f023001c9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f655f54f3166f6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:48:48.738 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.753+0000 7f0235d9b640 1 -- 172.21.15.80:0/4197335776 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02300a5780 con 0x7f02380b2530 2024-04-25T23:48:48.738 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.753+0000 7f0235d9b640 1 -- 172.21.15.80:0/4197335776 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02300a5920 con 0x7f02380b2530 2024-04-25T23:48:48.738 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.753+0000 7f0235d9b640 1 -- 172.21.15.80:0/4197335776 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02300a5cf0 con 0x7f02380b2530 2024-04-25T23:48:48.739 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.753+0000 7f023d1e8640 1 -- 172.21.15.80:0/4197335776 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02380b2530 msgr2=0x7f023814cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:48.739 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.753+0000 7f023d1e8640 1 --2- 172.21.15.80:0/4197335776 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02380b2530 0x7f023814cf10 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f02380aa230 tx=0x7f023001c9d0 comp rx=0 tx=0).stop 2024-04-25T23:48:48.739 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.753+0000 7f023d1e8640 1 -- 172.21.15.80:0/4197335776 shutdown_connections 2024-04-25T23:48:48.739 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.753+0000 7f023d1e8640 1 --2- 172.21.15.80:0/4197335776 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02380b2530 0x7f023814cf10 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:48.739 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.753+0000 7f023d1e8640 1 --2- 172.21.15.80:0/4197335776 >> v2:172.21.15.57:3300/0 conn(0x7f02380b54f0 0x7f0238146010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:48.739 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.753+0000 7f023d1e8640 1 -- 172.21.15.80:0/4197335776 >> 172.21.15.80:0/4197335776 conn(0x7f0238074cf0 msgr2=0x7f0238075100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:48:48.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.753+0000 7f023d1e8640 1 -- 172.21.15.80:0/4197335776 shutdown_connections 2024-04-25T23:48:48.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.753+0000 7f023d1e8640 1 -- 172.21.15.80:0/4197335776 wait complete. 2024-04-25T23:48:48.742 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.753+0000 7f023d1e8640 1 Processor -- start 2024-04-25T23:48:48.742 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.753+0000 7f023d1e8640 1 -- start start 2024-04-25T23:48:48.742 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.753+0000 7f023d1e8640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f02380b2530 0x7f023813a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:48.742 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.753+0000 7f023d1e8640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f02380b54f0 0x7f0238143c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:48.742 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.753+0000 7f023d1e8640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0238146550 0x7f02381ddd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:48.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.753+0000 7f023d1e8640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f02380a9930 con 0x7f02380b54f0 2024-04-25T23:48:48.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.753+0000 7f023d1e8640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f02380a97b0 con 0x7f0238146550 2024-04-25T23:48:48.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.753+0000 7f023d1e8640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f02380aa2a0 con 0x7f02380b2530 2024-04-25T23:48:48.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.753+0000 7f023759e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0238146550 0x7f02381ddd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:48.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.753+0000 7f023759e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0238146550 0x7f02381ddd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:59222/0 (socket says 172.21.15.80:59222) 2024-04-25T23:48:48.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.753+0000 7f023759e640 1 -- 172.21.15.80:0/2305455274 learned_addr learned my addr 172.21.15.80:0/2305455274 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T23:48:48.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.753+0000 7f023659c640 1 --2- 172.21.15.80:0/2305455274 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f02380b54f0 0x7f0238143c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:48.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.753+0000 7f0236d9d640 1 --2- 172.21.15.80:0/2305455274 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f02380b2530 0x7f023813a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:48.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.757+0000 7f023759e640 1 -- 172.21.15.80:0/2305455274 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f02380b2530 msgr2=0x7f023813a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:48.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.757+0000 7f023759e640 1 --2- 172.21.15.80:0/2305455274 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f02380b2530 0x7f023813a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:48.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.757+0000 7f023759e640 1 -- 172.21.15.80:0/2305455274 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f02380b54f0 msgr2=0x7f0238143c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:48.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.757+0000 7f023759e640 1 --2- 172.21.15.80:0/2305455274 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f02380b54f0 0x7f0238143c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:48.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.757+0000 7f023759e640 1 -- 172.21.15.80:0/2305455274 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0238145650 con 0x7f0238146550 2024-04-25T23:48:48.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.757+0000 7f0236d9d640 1 --2- 172.21.15.80:0/2305455274 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f02380b2530 0x7f023813a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T23:48:48.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.757+0000 7f023759e640 1 --2- 172.21.15.80:0/2305455274 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0238146550 0x7f02381ddd10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f023001a600 tx=0x7f0230019770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:48:48.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.757+0000 7f021f7fe640 1 -- 172.21.15.80:0/2305455274 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02300bc020 con 0x7f0238146550 2024-04-25T23:48:48.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.757+0000 7f023d1e8640 1 -- 172.21.15.80:0/2305455274 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02381458c0 con 0x7f0238146550 2024-04-25T23:48:48.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.757+0000 7f021f7fe640 1 -- 172.21.15.80:0/2305455274 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02300b8690 con 0x7f0238146550 2024-04-25T23:48:48.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.757+0000 7f021f7fe640 1 -- 172.21.15.80:0/2305455274 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02300b8a80 con 0x7f0238146550 2024-04-25T23:48:48.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.757+0000 7f023d1e8640 1 -- 172.21.15.80:0/2305455274 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02380abef0 con 0x7f0238146550 2024-04-25T23:48:48.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.757+0000 7f021f7fe640 1 -- 172.21.15.80:0/2305455274 <== mon.1 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f02300bf020 con 0x7f0238146550 2024-04-25T23:48:48.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.757+0000 7f021f7fe640 1 -- 172.21.15.80:0/2305455274 <== mon.1 v2:172.21.15.80:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f02300c1d70 con 0x7f0238146550 2024-04-25T23:48:48.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.757+0000 7f023d1e8640 1 -- 172.21.15.80:0/2305455274 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0238068a10 con 0x7f0238146550 2024-04-25T23:48:48.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.757+0000 7f021f7fe640 1 -- 172.21.15.80:0/2305455274 <== mon.1 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f02300165c0 con 0x7f0238146550 2024-04-25T23:48:48.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.833+0000 7f023d1e8640 1 -- 172.21.15.80:0/2305455274 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd new", "uuid": "0e9d533c-465f-403c-8fe8-7dafc953d774", "id": 7} v 0) -- 0x7f02380b4170 con 0x7f0238146550 2024-04-25T23:48:48.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.841+0000 7f021f7fe640 1 -- 172.21.15.80:0/2305455274 <== mon.1 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0e9d533c-465f-403c-8fe8-7dafc953d774", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f02300b6af0 con 0x7f0238146550 2024-04-25T23:48:48.826 INFO:teuthology.orchestra.run.smithi080.stdout:7 2024-04-25T23:48:48.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.841+0000 7f023d1e8640 1 -- 172.21.15.80:0/2305455274 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0238146550 msgr2=0x7f02381ddd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:48.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.841+0000 7f023d1e8640 1 --2- 172.21.15.80:0/2305455274 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0238146550 0x7f02381ddd10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f023001a600 tx=0x7f0230019770 comp rx=0 tx=0).stop 2024-04-25T23:48:48.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.841+0000 7f023d1e8640 1 -- 172.21.15.80:0/2305455274 shutdown_connections 2024-04-25T23:48:48.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.841+0000 7f023d1e8640 1 --2- 172.21.15.80:0/2305455274 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0238146550 0x7f02381ddd10 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:48.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.841+0000 7f023d1e8640 1 --2- 172.21.15.80:0/2305455274 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f02380b54f0 0x7f0238143c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:48.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.841+0000 7f023d1e8640 1 --2- 172.21.15.80:0/2305455274 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f02380b2530 0x7f023813a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:48.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.841+0000 7f023d1e8640 1 -- 172.21.15.80:0/2305455274 >> 172.21.15.80:0/2305455274 conn(0x7f0238074cf0 msgr2=0x7f02380b5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:48:48.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.841+0000 7f023d1e8640 1 -- 172.21.15.80:0/2305455274 shutdown_connections 2024-04-25T23:48:48.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T23:48:48.841+0000 7f023d1e8640 1 -- 172.21.15.80:0/2305455274 wait complete. 2024-04-25T23:48:48.843 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-25T23:48:48.843 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-25T23:48:48.846 INFO:tasks.ceph.osd.0:Started 2024-04-25T23:48:48.846 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-25T23:48:48.846 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-25T23:48:48.848 INFO:tasks.ceph.osd.1:Started 2024-04-25T23:48:48.848 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-25T23:48:48.848 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-25T23:48:48.850 INFO:tasks.ceph.osd.2:Started 2024-04-25T23:48:48.851 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-25T23:48:48.851 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-25T23:48:48.853 INFO:tasks.ceph.osd.3:Started 2024-04-25T23:48:48.853 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-25T23:48:48.854 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-25T23:48:48.856 INFO:tasks.ceph.osd.4:Started 2024-04-25T23:48:48.856 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-25T23:48:48.856 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-25T23:48:48.858 INFO:tasks.ceph.osd.5:Started 2024-04-25T23:48:48.858 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-25T23:48:48.858 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-25T23:48:48.860 INFO:tasks.ceph.osd.6:Started 2024-04-25T23:48:48.860 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-25T23:48:48.861 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-25T23:48:48.862 INFO:tasks.ceph.osd.7:Started 2024-04-25T23:48:48.863 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T23:48:48.942 INFO:tasks.ceph.mgr.x.smithi080.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T23:48:48.943 INFO:tasks.ceph.mgr.x.smithi080.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T23:48:48.943 INFO:tasks.ceph.mgr.x.smithi080.stderr: from numpy import show_config as show_numpy_config 2024-04-25T23:48:48.984 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-04-25T23:48:48.997+0000 7f625a234180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T23:48:48.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:48.995+0000 7fc2f83c7640 1 Processor -- start 2024-04-25T23:48:48.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:48.995+0000 7fc2f83c7640 1 -- start start 2024-04-25T23:48:48.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:48.999+0000 7fc2f83c7640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fc2f013f950 0x7fc2f013c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:48.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:48.999+0000 7fc2f83c7640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fc2f013c680 0x7fc2f0149b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:48.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:48.999+0000 7fc2f83c7640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2f0072610 con 0x7fc2f0073050 2024-04-25T23:48:48.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:48.999+0000 7fc2f83c7640 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 -- 0x7fc2f00728f0 con 0x7fc2f013c680 2024-04-25T23:48:48.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:48.999+0000 7fc2f83c7640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7fc2f0071780 con 0x7fc2f013f950 2024-04-25T23:48:48.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:48.999+0000 7fc2f5962640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fc2f013f950 0x7fc2f013c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:48.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:48.999+0000 7fc2f5962640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fc2f013f950 0x7fc2f013c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3301/0 says I am v2:172.21.15.57:36192/0 (socket says 172.21.15.57:36192) 2024-04-25T23:48:48.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:48.999+0000 7fc2f5962640 1 -- 172.21.15.57:0/2702173096 learned_addr learned my addr 172.21.15.57:0/2702173096 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:48:48.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:48.999+0000 7fc2f6964640 1 --2- 172.21.15.57:0/2702173096 >> v2:172.21.15.57:3300/0 conn(0x7fc2f013c680 0x7fc2f0149b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:48.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:48.999+0000 7fc2f5962640 1 -- 172.21.15.57:0/2702173096 >> v2:172.21.15.57:3300/0 conn(0x7fc2f013c680 msgr2=0x7fc2f0149b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:48:48.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:48.999+0000 7fc2f5962640 1 --2- 172.21.15.57:0/2702173096 >> v2:172.21.15.57:3300/0 conn(0x7fc2f013c680 0x7fc2f0149b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:48.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:48.999+0000 7fc2f5962640 1 -- 172.21.15.57:0/2702173096 >> v1:172.21.15.80:6789/0 conn(0x7fc2f0073050 legacy=0x7fc2f013f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:48.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:48.999+0000 7fc2f5962640 1 -- 172.21.15.57:0/2702173096 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2f013cd80 con 0x7fc2f013f950 2024-04-25T23:48:48.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:48.999+0000 7fc2f5962640 1 --2- 172.21.15.57:0/2702173096 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fc2f013f950 0x7fc2f013c140 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc2e000bb10 tx=0x7fc2e001ec10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3beed9ee0c908992 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:48:48.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:48.999+0000 7fc2f5161640 1 -- 172.21.15.57:0/2702173096 <== mon.2 v2:172.21.15.57:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e00b4070 con 0x7fc2f013f950 2024-04-25T23:48:48.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:48.999+0000 7fc2f5161640 1 -- 172.21.15.57:0/2702173096 <== mon.2 v2:172.21.15.57:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2e00a7af0 con 0x7fc2f013f950 2024-04-25T23:48:48.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:48.999+0000 7fc2f5161640 1 -- 172.21.15.57:0/2702173096 <== mon.2 v2:172.21.15.57:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e00af440 con 0x7fc2f013f950 2024-04-25T23:48:48.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:48.999+0000 7fc2f83c7640 1 -- 172.21.15.57:0/2702173096 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fc2f013f950 msgr2=0x7fc2f013c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:48.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:48.999+0000 7fc2f83c7640 1 --2- 172.21.15.57:0/2702173096 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fc2f013f950 0x7fc2f013c140 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc2e000bb10 tx=0x7fc2e001ec10 comp rx=0 tx=0).stop 2024-04-25T23:48:48.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:48.999+0000 7fc2f83c7640 1 -- 172.21.15.57:0/2702173096 shutdown_connections 2024-04-25T23:48:48.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:48.999+0000 7fc2f83c7640 1 --2- 172.21.15.57:0/2702173096 >> v2:172.21.15.57:3300/0 conn(0x7fc2f013c680 0x7fc2f0149b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:48.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:48.999+0000 7fc2f83c7640 1 --2- 172.21.15.57:0/2702173096 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fc2f013f950 0x7fc2f013c140 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:48.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:48.999+0000 7fc2f83c7640 1 -- 172.21.15.57:0/2702173096 >> 172.21.15.57:0/2702173096 conn(0x7fc2f00b5320 msgr2=0x7fc2f00b5730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:48:48.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:48.999+0000 7fc2f83c7640 1 -- 172.21.15.57:0/2702173096 shutdown_connections 2024-04-25T23:48:48.989 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:48.999+0000 7fc2f83c7640 1 -- 172.21.15.57:0/2702173096 wait complete. 2024-04-25T23:48:48.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:49.003+0000 7fc2f83c7640 1 Processor -- start 2024-04-25T23:48:48.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:49.003+0000 7fc2f83c7640 1 -- start start 2024-04-25T23:48:48.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:49.003+0000 7fc2f83c7640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fc2f0073050 0x7fc2f0142360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:48.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:49.003+0000 7fc2f83c7640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc2f013c680 0x7fc2f0145460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:48.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:49.003+0000 7fc2f83c7640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2f0144ae0 0x7fc2f01d6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:48.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:49.003+0000 7fc2f83c7640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f0075280 con 0x7fc2f013c680 2024-04-25T23:48:48.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:49.003+0000 7fc2f83c7640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f0075100 con 0x7fc2f0144ae0 2024-04-25T23:48:48.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:49.003+0000 7fc2f83c7640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7fc2f0075400 con 0x7fc2f0073050 2024-04-25T23:48:48.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:49.003+0000 7fc2f6163640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fc2f0073050 0x7fc2f0142360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:48.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:49.003+0000 7fc2f5962640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc2f013c680 0x7fc2f0145460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:48.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:49.003+0000 7fc2f6163640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fc2f0073050 0x7fc2f0142360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3301/0 says I am v2:172.21.15.57:36200/0 (socket says 172.21.15.57:36200) 2024-04-25T23:48:48.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:49.003+0000 7fc2f6163640 1 -- 172.21.15.57:0/2791457864 learned_addr learned my addr 172.21.15.57:0/2791457864 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:48:48.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:49.003+0000 7fc2f6964640 1 --2- 172.21.15.57:0/2791457864 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2f0144ae0 0x7fc2f01d6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:48.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:49.003+0000 7fc2f5962640 1 -- 172.21.15.57:0/2791457864 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fc2f0073050 msgr2=0x7fc2f0142360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:48.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:49.003+0000 7fc2f5962640 1 --2- 172.21.15.57:0/2791457864 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fc2f0073050 0x7fc2f0142360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:48.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:49.003+0000 7fc2f5962640 1 -- 172.21.15.57:0/2791457864 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2f0144ae0 msgr2=0x7fc2f01d6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:48.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:49.003+0000 7fc2f5962640 1 --2- 172.21.15.57:0/2791457864 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2f0144ae0 0x7fc2f01d6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:48.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:49.003+0000 7fc2f5962640 1 -- 172.21.15.57:0/2791457864 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2f0145cc0 con 0x7fc2f013c680 2024-04-25T23:48:48.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:49.003+0000 7fc2f6163640 1 --2- 172.21.15.57:0/2791457864 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fc2f0073050 0x7fc2f0142360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T23:48:48.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:49.003+0000 7fc2f5962640 1 --2- 172.21.15.57:0/2791457864 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc2f013c680 0x7fc2f0145460 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc2e00ae000 tx=0x7fc2e001b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:48:48.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:49.003+0000 7fc2e6ffd640 1 -- 172.21.15.57:0/2791457864 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e00be020 con 0x7fc2f013c680 2024-04-25T23:48:48.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:49.003+0000 7fc2e6ffd640 1 -- 172.21.15.57:0/2791457864 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2e00afc70 con 0x7fc2f013c680 2024-04-25T23:48:48.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:49.003+0000 7fc2e6ffd640 1 -- 172.21.15.57:0/2791457864 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e00b4040 con 0x7fc2f013c680 2024-04-25T23:48:48.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:49.003+0000 7fc2f83c7640 1 -- 172.21.15.57:0/2791457864 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2f01d6a90 con 0x7fc2f013c680 2024-04-25T23:48:48.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:49.003+0000 7fc2f83c7640 1 -- 172.21.15.57:0/2791457864 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2f01d6f50 con 0x7fc2f013c680 2024-04-25T23:48:48.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:49.003+0000 7fc2f5962640 1 -- 172.21.15.57:0/2791457864 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc2f013c680 msgr2=0x7fc2f0145460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-04-25T23:48:48.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:49.003+0000 7fc2f5962640 1 -- 172.21.15.57:0/2791457864 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc2f013c680 msgr2=0x7fc2f0145460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T23:48:48.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:49.003+0000 7fc2f5962640 1 --2- 172.21.15.57:0/2791457864 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc2f013c680 0x7fc2f0145460 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc2e00ae000 tx=0x7fc2e001b9b0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T23:48:48.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:49.003+0000 7fc2f5962640 1 --2- 172.21.15.57:0/2791457864 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc2f013c680 0x7fc2f0145460 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc2e00ae000 tx=0x7fc2e001b9b0 comp rx=0 tx=0).stop 2024-04-25T23:48:48.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:49.003+0000 7fc2e6ffd640 1 -- 172.21.15.57:0/2791457864 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc2f013c680 msgr2=0x7fc2f0145460 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T23:48:48.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:49.003+0000 7fc2e6ffd640 1 --2- 172.21.15.57:0/2791457864 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc2f013c680 0x7fc2f0145460 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:48.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:49.003+0000 7fc2e6ffd640 1 --2- 172.21.15.57:0/2791457864 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2f0144ae0 0x7fc2d4005880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:48.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:49.003+0000 7fc2e6ffd640 1 --2- 172.21.15.57:0/2791457864 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fc2f0073050 0x7fc2d4008280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:48.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:49.003+0000 7fc2e6ffd640 1 --2- 172.21.15.57:0/2791457864 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc2d4008c90 0x7fc2d400b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:48.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:49.003+0000 7fc2e6ffd640 1 -- 172.21.15.57:0/2791457864 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d400bbd0 con 0x7fc2d4008c90 2024-04-25T23:48:48.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:49.003+0000 7fc2e6ffd640 1 -- 172.21.15.57:0/2791457864 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d400bd50 con 0x7fc2f0144ae0 2024-04-25T23:48:48.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:49.003+0000 7fc2e6ffd640 1 -- 172.21.15.57:0/2791457864 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7fc2d400bed0 con 0x7fc2f0073050 2024-04-25T23:48:48.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:49.003+0000 7fc2f6964640 1 --2- 172.21.15.57:0/2791457864 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc2d4008c90 0x7fc2d400b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:48.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:49.003+0000 7fc2f5962640 1 --2- 172.21.15.57:0/2791457864 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fc2f0073050 0x7fc2d4008280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:48.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:49.003+0000 7fc2f6163640 1 --2- 172.21.15.57:0/2791457864 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2f0144ae0 0x7fc2d4005880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:48.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:49.003+0000 7fc2f6964640 1 -- 172.21.15.57:0/2791457864 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fc2f0073050 msgr2=0x7fc2d4008280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:48.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:49.003+0000 7fc2f6964640 1 --2- 172.21.15.57:0/2791457864 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fc2f0073050 0x7fc2d4008280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:48.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:49.003+0000 7fc2f6964640 1 -- 172.21.15.57:0/2791457864 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2f0144ae0 msgr2=0x7fc2d4005880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:48.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:49.003+0000 7fc2f6964640 1 --2- 172.21.15.57:0/2791457864 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2f0144ae0 0x7fc2d4005880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:48.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:49.003+0000 7fc2f6964640 1 -- 172.21.15.57:0/2791457864 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,mgrmap=0+,monmap=2+,osdmap=0}) -- 0x7fc2d400c200 con 0x7fc2d4008c90 2024-04-25T23:48:48.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:49.003+0000 7fc2f6964640 1 -- 172.21.15.57:0/2791457864 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2f0068210 con 0x7fc2d4008c90 2024-04-25T23:48:48.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:49.003+0000 7fc2f6964640 1 -- 172.21.15.57:0/2791457864 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2e8007770 con 0x7fc2d4008c90 2024-04-25T23:48:48.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:49.003+0000 7fc2f5962640 1 --2- 172.21.15.57:0/2791457864 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fc2f0073050 0x7fc2d4008280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T23:48:48.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:49.003+0000 7fc2f6964640 1 --2- 172.21.15.57:0/2791457864 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc2d4008c90 0x7fc2d400b120 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc2e80079a0 tx=0x7fc2e00a9070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:48:48.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:49.003+0000 7fc2e6ffd640 1 -- 172.21.15.57:0/2791457864 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e8009d90 con 0x7fc2d4008c90 2024-04-25T23:48:48.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:49.003+0000 7fc2e6ffd640 1 -- 172.21.15.57:0/2791457864 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2e801b070 con 0x7fc2d4008c90 2024-04-25T23:48:48.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:49.003+0000 7fc2e6ffd640 1 -- 172.21.15.57:0/2791457864 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc2e8011a30 con 0x7fc2d4008c90 2024-04-25T23:48:48.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:49.003+0000 7fc2e6ffd640 1 -- 172.21.15.57:0/2791457864 <== mon.0 v2:172.21.15.57:3300/0 4 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fc2e801faf0 con 0x7fc2d4008c90 2024-04-25T23:48:48.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:49.011+0000 7fc2e6ffd640 1 -- 172.21.15.57:0/2791457864 <== mon.0 v2:172.21.15.57:3300/0 5 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fc2e801fcf0 con 0x7fc2d4008c90 2024-04-25T23:48:48.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:49.011+0000 7fc2e6ffd640 1 -- 172.21.15.57:0/2791457864 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fc2e8011d20 con 0x7fc2d4008c90 2024-04-25T23:48:49.056 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:49.067+0000 7fc2f83c7640 1 -- 172.21.15.57:0/2791457864 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc2f01d71e0 con 0x7fc2d4008c90 2024-04-25T23:48:49.057 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:49.071+0000 7fc2e6ffd640 1 -- 172.21.15.57:0/2791457864 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fc2e801f420 con 0x7fc2d4008c90 2024-04-25T23:48:49.057 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-25T23:48:49.057 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":10,"fsid":"ca0adda3-98dd-4ba5-b4b2-8b4f20dcabdd","created":"2024-04-25T23:48:46.211894+0000","modified":"2024-04-25T23:48:48.838840+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T23:48:48.838840+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"5986c317-8828-4568-af68-a8ae96690c51","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"1aa157dd-52fe-4bb3-9f63-4db493de5333","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"1391be38-25ac-43f0-8090-15adddc6d69d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"a85e7650-effb-4ea7-9f3b-9fc9fc1576e4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ad3fb1a9-cec6-4fb3-9f49-22b9480f524b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"fd05cda6-27a5-477a-8916-b34b69ba9ef6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"599ef421-483d-4225-a98f-7405006b1809","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"0e9d533c-465f-403c-8fe8-7dafc953d774","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T23:48:49.058 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:49.071+0000 7fc2f83c7640 1 -- 172.21.15.57:0/2791457864 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc2d4008c90 msgr2=0x7fc2d400b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:49.059 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:49.071+0000 7fc2f83c7640 1 --2- 172.21.15.57:0/2791457864 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc2d4008c90 0x7fc2d400b120 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc2e80079a0 tx=0x7fc2e00a9070 comp rx=0 tx=0).stop 2024-04-25T23:48:49.059 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:49.071+0000 7fc2f83c7640 1 -- 172.21.15.57:0/2791457864 shutdown_connections 2024-04-25T23:48:49.059 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:49.071+0000 7fc2f83c7640 1 --2- 172.21.15.57:0/2791457864 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2f0144ae0 0x7fc2d4005880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:49.059 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:49.071+0000 7fc2f83c7640 1 --2- 172.21.15.57:0/2791457864 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc2d4008c90 0x7fc2d400b120 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:49.059 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:49.071+0000 7fc2f83c7640 1 --2- 172.21.15.57:0/2791457864 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fc2f0073050 0x7fc2d4008280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:49.059 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:49.071+0000 7fc2f83c7640 1 -- 172.21.15.57:0/2791457864 >> 172.21.15.57:0/2791457864 conn(0x7fc2f00b5320 msgr2=0x7fc2f00b20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:48:49.059 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:49.071+0000 7fc2f83c7640 1 -- 172.21.15.57:0/2791457864 shutdown_connections 2024-04-25T23:48:49.059 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:49.071+0000 7fc2f83c7640 1 -- 172.21.15.57:0/2791457864 wait complete. 2024-04-25T23:48:49.076 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-25T23:48:49.076 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-25T23:48:49.089 INFO:tasks.ceph.mgr.y.smithi057.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T23:48:49.089 INFO:tasks.ceph.mgr.y.smithi057.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T23:48:49.089 INFO:tasks.ceph.mgr.y.smithi057.stderr: from numpy import show_config as show_numpy_config 2024-04-25T23:48:49.116 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-04-25T23:48:49.129+0000 7f625a234180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T23:48:49.124 INFO:tasks.ceph.mgr.y.smithi057.stderr:2024-04-25T23:48:49.135+0000 7f7104d7b180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T23:48:49.274 INFO:tasks.ceph.mgr.y.smithi057.stderr:2024-04-25T23:48:49.287+0000 7f7104d7b180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T23:48:49.281 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-04-25T23:48:49.297+0000 7f625a234180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T23:48:49.344 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-04-25T23:48:49.357+0000 7f625a234180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T23:48:49.438 INFO:tasks.ceph.mgr.y.smithi057.stderr:2024-04-25T23:48:49.451+0000 7f7104d7b180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T23:48:49.451 INFO:tasks.ceph.osd.3.smithi057.stderr:2024-04-25T23:48:49.463+0000 7f396fcdb7c0 -1 Falling back to public interface 2024-04-25T23:48:49.466 INFO:tasks.ceph.osd.2.smithi057.stderr:2024-04-25T23:48:49.479+0000 7f2328e027c0 -1 Falling back to public interface 2024-04-25T23:48:49.470 INFO:tasks.ceph.osd.4.smithi080.stderr:2024-04-25T23:48:49.485+0000 7f96503e77c0 -1 Falling back to public interface 2024-04-25T23:48:49.470 INFO:tasks.ceph.osd.6.smithi080.stderr:2024-04-25T23:48:49.485+0000 7fa39e19c7c0 -1 Falling back to public interface 2024-04-25T23:48:49.474 INFO:tasks.ceph.osd.1.smithi057.stderr:2024-04-25T23:48:49.487+0000 7f9c4c9bb7c0 -1 Falling back to public interface 2024-04-25T23:48:49.478 INFO:tasks.ceph.osd.0.smithi057.stderr:2024-04-25T23:48:49.491+0000 7fb8944247c0 -1 Falling back to public interface 2024-04-25T23:48:49.478 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-04-25T23:48:49.493+0000 7f625a234180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T23:48:49.481 INFO:tasks.ceph.osd.7.smithi080.stderr:2024-04-25T23:48:49.497+0000 7f65e710a7c0 -1 Falling back to public interface 2024-04-25T23:48:49.482 INFO:tasks.ceph.osd.5.smithi080.stderr:2024-04-25T23:48:49.497+0000 7f286957a7c0 -1 Falling back to public interface 2024-04-25T23:48:49.509 INFO:tasks.ceph.mgr.y.smithi057.stderr:2024-04-25T23:48:49.523+0000 7f7104d7b180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T23:48:49.651 INFO:tasks.ceph.mgr.y.smithi057.stderr:2024-04-25T23:48:49.663+0000 7f7104d7b180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T23:48:49.881 DEBUG:teuthology.orchestra.run.smithi057:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T23:48:49.997 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:50.007+0000 7fc5ae513640 1 Processor -- start 2024-04-25T23:48:49.997 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:50.007+0000 7fc5ae513640 1 -- start start 2024-04-25T23:48:49.997 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:50.007+0000 7fc5ae513640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fc5a806d170 0x7fc5a80b0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:49.998 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:50.007+0000 7fc5ae513640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5a80715e0 con 0x7fc5a806cd90 2024-04-25T23:48:49.998 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:50.007+0000 7fc5ae513640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5a806e320 con 0x7fc5a813fa50 2024-04-25T23:48:49.998 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:50.007+0000 7fc5ae513640 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 -- 0x7fc5a8142ce0 con 0x7fc5a806d170 2024-04-25T23:48:49.998 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:50.007+0000 7fc5a77fe640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fc5a806d170 0x7fc5a80b0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:49.998 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:50.007+0000 7fc5a7fff640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fc5a806cd90 0x7fc5a8143140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:46876/0 (socket says 172.21.15.57:46876) 2024-04-25T23:48:49.998 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:50.007+0000 7fc5a7fff640 1 -- 172.21.15.57:0/533532438 learned_addr learned my addr 172.21.15.57:0/533532438 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-04-25T23:48:49.998 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:50.007+0000 7fc5a6ffd640 1 -- 172.21.15.57:0/533532438 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4189178156 0 0) 0x7fc5a80715e0 con 0x7fc5a806cd90 2024-04-25T23:48:49.998 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:50.007+0000 7fc5a6ffd640 1 -- 172.21.15.57:0/533532438 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc590003530 con 0x7fc5a806cd90 2024-04-25T23:48:49.998 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:50.007+0000 7fc5a77fe640 1 -- 172.21.15.57:0/533532438 >> v1:172.21.15.80:6789/0 conn(0x7fc5a813fa50 legacy=0x7fc5a813fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:49.998 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:50.007+0000 7fc5a77fe640 1 -- 172.21.15.57:0/533532438 >> v1:172.21.15.57:6789/0 conn(0x7fc5a806cd90 legacy=0x7fc5a8143140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:49.998 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:50.007+0000 7fc5a77fe640 1 -- 172.21.15.57:0/533532438 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5a806d5e0 con 0x7fc5a806d170 2024-04-25T23:48:50.000 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:50.011+0000 7fc5a77fe640 1 --2- 172.21.15.57:0/533532438 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5a806d170 0x7fc5a80b0ed0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc59c0097c0 tx=0x7fc59c01c320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e98fe162241643b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:48:50.000 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:50.011+0000 7fc5a6ffd640 1 -- 172.21.15.57:0/533532438 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59c0b0070 con 0x7fc5a806d170 2024-04-25T23:48:50.000 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:50.011+0000 7fc5a6ffd640 1 -- 172.21.15.57:0/533532438 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc59c0ab600 con 0x7fc5a806d170 2024-04-25T23:48:50.000 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:50.011+0000 7fc5a6ffd640 1 -- 172.21.15.57:0/533532438 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59c0ab9b0 con 0x7fc5a806d170 2024-04-25T23:48:50.000 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:50.011+0000 7fc5ae513640 1 -- 172.21.15.57:0/533532438 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5a806d170 msgr2=0x7fc5a80b0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:50.000 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:50.011+0000 7fc5ae513640 1 --2- 172.21.15.57:0/533532438 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5a806d170 0x7fc5a80b0ed0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc59c0097c0 tx=0x7fc59c01c320 comp rx=0 tx=0).stop 2024-04-25T23:48:50.001 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:50.011+0000 7fc5ae513640 1 -- 172.21.15.57:0/533532438 shutdown_connections 2024-04-25T23:48:50.001 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:50.011+0000 7fc5ae513640 1 --2- 172.21.15.57:0/533532438 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5a806d170 0x7fc5a80b0ed0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:50.001 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:50.011+0000 7fc5ae513640 1 -- 172.21.15.57:0/533532438 >> 172.21.15.57:0/533532438 conn(0x7fc5a80b65c0 msgr2=0x7fc5a80b69d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:48:50.001 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:50.011+0000 7fc5ae513640 1 -- 172.21.15.57:0/533532438 shutdown_connections 2024-04-25T23:48:50.001 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:50.011+0000 7fc5ae513640 1 -- 172.21.15.57:0/533532438 wait complete. 2024-04-25T23:48:50.001 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:50.015+0000 7fc5ae513640 1 Processor -- start 2024-04-25T23:48:50.002 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:50.015+0000 7fc5ae513640 1 -- start start 2024-04-25T23:48:50.005 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:50.015+0000 7fc5ae513640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5a806cd90 0x7fc5a80af150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:50.005 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:50.015+0000 7fc5ae513640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc5a806d170 0x7fc5a80b0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:50.005 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:50.015+0000 7fc5ae513640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fc5a813fa50 0x7fc5a80ad860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:50.005 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:50.015+0000 7fc5ae513640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a8150c20 con 0x7fc5a806cd90 2024-04-25T23:48:50.005 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:50.015+0000 7fc5ae513640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a8150aa0 con 0x7fc5a806d170 2024-04-25T23:48:50.005 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:50.015+0000 7fc5ae513640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7fc5a8150da0 con 0x7fc5a813fa50 2024-04-25T23:48:50.005 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:50.015+0000 7fc5a7fff640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5a806cd90 0x7fc5a80af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:50.005 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:50.015+0000 7fc5a7fff640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5a806cd90 0x7fc5a80af150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44538/0 (socket says 172.21.15.57:44538) 2024-04-25T23:48:50.005 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:50.015+0000 7fc5a7fff640 1 -- 172.21.15.57:0/917521696 learned_addr learned my addr 172.21.15.57:0/917521696 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:48:50.005 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:50.015+0000 7fc5acab0640 1 --2- 172.21.15.57:0/917521696 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fc5a813fa50 0x7fc5a80ad860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:50.005 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:50.015+0000 7fc5a77fe640 1 --2- 172.21.15.57:0/917521696 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc5a806d170 0x7fc5a80b0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:50.005 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:50.015+0000 7fc5a7fff640 1 -- 172.21.15.57:0/917521696 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fc5a813fa50 msgr2=0x7fc5a80ad860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:50.006 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:50.015+0000 7fc5a7fff640 1 --2- 172.21.15.57:0/917521696 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fc5a813fa50 0x7fc5a80ad860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:50.006 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:50.015+0000 7fc5a7fff640 1 -- 172.21.15.57:0/917521696 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc5a806d170 msgr2=0x7fc5a80b0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:50.006 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:50.015+0000 7fc5a7fff640 1 --2- 172.21.15.57:0/917521696 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc5a806d170 0x7fc5a80b0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:50.006 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:50.015+0000 7fc5a7fff640 1 -- 172.21.15.57:0/917521696 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5a8135dd0 con 0x7fc5a806cd90 2024-04-25T23:48:50.006 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:50.015+0000 7fc5acab0640 1 --2- 172.21.15.57:0/917521696 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fc5a813fa50 0x7fc5a80ad860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T23:48:50.006 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:50.015+0000 7fc5a7fff640 1 --2- 172.21.15.57:0/917521696 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5a806cd90 0x7fc5a80af150 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc59800c520 tx=0x7fc59800c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:48:50.006 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:50.015+0000 7fc5a4ff9640 1 -- 172.21.15.57:0/917521696 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc598018070 con 0x7fc5a806cd90 2024-04-25T23:48:50.006 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:50.015+0000 7fc5a4ff9640 1 -- 172.21.15.57:0/917521696 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5980074e0 con 0x7fc5a806cd90 2024-04-25T23:48:50.006 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:50.015+0000 7fc5ae513640 1 -- 172.21.15.57:0/917521696 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5a8136040 con 0x7fc5a806cd90 2024-04-25T23:48:50.006 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:50.015+0000 7fc5a4ff9640 1 -- 172.21.15.57:0/917521696 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5980049f0 con 0x7fc5a806cd90 2024-04-25T23:48:50.006 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:50.015+0000 7fc5ae513640 1 -- 172.21.15.57:0/917521696 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5a80af790 con 0x7fc5a806cd90 2024-04-25T23:48:50.006 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:50.015+0000 7fc5a4ff9640 1 -- 172.21.15.57:0/917521696 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc59801b070 con 0x7fc5a806cd90 2024-04-25T23:48:50.006 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:50.015+0000 7fc5a4ff9640 1 -- 172.21.15.57:0/917521696 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fc598012ad0 con 0x7fc5a806cd90 2024-04-25T23:48:50.006 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:50.015+0000 7fc5ae513640 1 -- 172.21.15.57:0/917521696 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5a80afa20 con 0x7fc5a806cd90 2024-04-25T23:48:50.007 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:50.019+0000 7fc5a4ff9640 1 -- 172.21.15.57:0/917521696 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fc598022080 con 0x7fc5a806cd90 2024-04-25T23:48:50.064 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:50.075+0000 7fc5ae513640 1 -- 172.21.15.57:0/917521696 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc5a804f510 con 0x7fc5a806cd90 2024-04-25T23:48:50.065 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:50.079+0000 7fc5a4ff9640 1 -- 172.21.15.57:0/917521696 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fc5a8136040 con 0x7fc5a806cd90 2024-04-25T23:48:50.066 INFO:teuthology.misc.health.smithi057.stdout: 2024-04-25T23:48:50.066 INFO:teuthology.misc.health.smithi057.stdout:{"epoch":10,"fsid":"ca0adda3-98dd-4ba5-b4b2-8b4f20dcabdd","created":"2024-04-25T23:48:46.211894+0000","modified":"2024-04-25T23:48:48.838840+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T23:48:48.838840+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"5986c317-8828-4568-af68-a8ae96690c51","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"1aa157dd-52fe-4bb3-9f63-4db493de5333","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"1391be38-25ac-43f0-8090-15adddc6d69d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"a85e7650-effb-4ea7-9f3b-9fc9fc1576e4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ad3fb1a9-cec6-4fb3-9f49-22b9480f524b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"fd05cda6-27a5-477a-8916-b34b69ba9ef6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"599ef421-483d-4225-a98f-7405006b1809","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"0e9d533c-465f-403c-8fe8-7dafc953d774","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T23:48:50.067 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:50.079+0000 7fc5ae513640 1 -- 172.21.15.57:0/917521696 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5a806cd90 msgr2=0x7fc5a80af150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:50.067 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:50.079+0000 7fc5ae513640 1 --2- 172.21.15.57:0/917521696 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5a806cd90 0x7fc5a80af150 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc59800c520 tx=0x7fc59800c9e0 comp rx=0 tx=0).stop 2024-04-25T23:48:50.067 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:50.079+0000 7fc5ae513640 1 -- 172.21.15.57:0/917521696 shutdown_connections 2024-04-25T23:48:50.067 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:50.079+0000 7fc5ae513640 1 --2- 172.21.15.57:0/917521696 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc5a806d170 0x7fc5a80b0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:50.067 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:50.079+0000 7fc5ae513640 1 --2- 172.21.15.57:0/917521696 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fc5a813fa50 0x7fc5a80ad860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:50.067 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:50.079+0000 7fc5ae513640 1 --2- 172.21.15.57:0/917521696 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5a806cd90 0x7fc5a80af150 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:50.067 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:50.079+0000 7fc5ae513640 1 -- 172.21.15.57:0/917521696 >> 172.21.15.57:0/917521696 conn(0x7fc5a80b65c0 msgr2=0x7fc5a80b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:48:50.067 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:50.079+0000 7fc5ae513640 1 -- 172.21.15.57:0/917521696 shutdown_connections 2024-04-25T23:48:50.067 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:50.079+0000 7fc5ae513640 1 -- 172.21.15.57:0/917521696 wait complete. 2024-04-25T23:48:50.081 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-25T23:48:50.090 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-04-25T23:48:50.105+0000 7f625a234180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T23:48:50.167 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-04-25T23:48:50.181+0000 7f625a234180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T23:48:50.298 INFO:tasks.ceph.mgr.y.smithi057.stderr:2024-04-25T23:48:50.311+0000 7f7104d7b180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T23:48:50.348 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-04-25T23:48:50.361+0000 7f625a234180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T23:48:50.378 INFO:tasks.ceph.mgr.y.smithi057.stderr:2024-04-25T23:48:50.391+0000 7f7104d7b180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T23:48:50.422 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-04-25T23:48:50.437+0000 7f625a234180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T23:48:50.567 INFO:tasks.ceph.mgr.y.smithi057.stderr:2024-04-25T23:48:50.579+0000 7f7104d7b180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T23:48:50.645 INFO:tasks.ceph.mgr.y.smithi057.stderr:2024-04-25T23:48:50.655+0000 7f7104d7b180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T23:48:50.710 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-04-25T23:48:50.725+0000 7f625a234180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T23:48:50.772 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-04-25T23:48:50.785+0000 7f625a234180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T23:48:50.954 INFO:tasks.ceph.mgr.y.smithi057.stderr:2024-04-25T23:48:50.967+0000 7f7104d7b180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T23:48:51.019 INFO:tasks.ceph.mgr.y.smithi057.stderr:2024-04-25T23:48:51.031+0000 7f7104d7b180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T23:48:51.817 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-04-25T23:48:51.833+0000 7f6201681640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-25T23:48:51.818 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-04-25T23:48:51.833+0000 7f6201681640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-25T23:48:53.014 INFO:tasks.ceph.osd.6.smithi080.stderr:2024-04-25T23:48:53.029+0000 7fa39e19c7c0 -1 osd.6 0 log_to_monitors true 2024-04-25T23:48:53.132 INFO:tasks.ceph.osd.5.smithi080.stderr:2024-04-25T23:48:53.145+0000 7f286957a7c0 -1 osd.5 0 log_to_monitors true 2024-04-25T23:48:53.146 INFO:tasks.ceph.osd.7.smithi080.stderr:2024-04-25T23:48:53.161+0000 7f65e710a7c0 -1 osd.7 0 log_to_monitors true 2024-04-25T23:48:53.257 INFO:tasks.ceph.osd.3.smithi057.stderr:2024-04-25T23:48:53.271+0000 7f396fcdb7c0 -1 osd.3 0 log_to_monitors true 2024-04-25T23:48:53.262 INFO:tasks.ceph.osd.4.smithi080.stderr:2024-04-25T23:48:53.277+0000 7f96503e77c0 -1 osd.4 0 log_to_monitors true 2024-04-25T23:48:53.365 INFO:tasks.ceph.osd.2.smithi057.stderr:2024-04-25T23:48:53.375+0000 7f2328e027c0 -1 osd.2 0 log_to_monitors true 2024-04-25T23:48:53.392 INFO:tasks.ceph.osd.1.smithi057.stderr:2024-04-25T23:48:53.403+0000 7f9c4c9bb7c0 -1 osd.1 0 log_to_monitors true 2024-04-25T23:48:53.472 INFO:tasks.ceph.osd.0.smithi057.stderr:2024-04-25T23:48:53.483+0000 7fb8944247c0 -1 osd.0 0 log_to_monitors true 2024-04-25T23:48:54.836 INFO:tasks.ceph.osd.0.smithi057.stderr:2024-04-25T23:48:54.847+0000 7fb8903cd640 -1 osd.0 0 waiting for initial osdmap 2024-04-25T23:48:54.836 INFO:tasks.ceph.osd.3.smithi057.stderr:2024-04-25T23:48:54.847+0000 7f396bc84640 -1 osd.3 0 waiting for initial osdmap 2024-04-25T23:48:54.838 INFO:tasks.ceph.osd.2.smithi057.stderr:2024-04-25T23:48:54.851+0000 7f2324dab640 -1 osd.2 0 waiting for initial osdmap 2024-04-25T23:48:54.838 INFO:tasks.ceph.osd.1.smithi057.stderr:2024-04-25T23:48:54.851+0000 7f9c48964640 -1 osd.1 0 waiting for initial osdmap 2024-04-25T23:48:54.844 INFO:tasks.ceph.osd.7.smithi080.stderr:2024-04-25T23:48:54.857+0000 7f65e30b3640 -1 osd.7 0 waiting for initial osdmap 2024-04-25T23:48:54.849 INFO:tasks.ceph.osd.2.smithi057.stderr:2024-04-25T23:48:54.863+0000 7f2320be8640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T23:48:54.853 INFO:tasks.ceph.osd.7.smithi080.stderr:2024-04-25T23:48:54.869+0000 7f65deef0640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T23:48:54.854 INFO:tasks.ceph.osd.1.smithi057.stderr:2024-04-25T23:48:54.867+0000 7f9c447a1640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T23:48:54.855 INFO:tasks.ceph.osd.0.smithi057.stderr:2024-04-25T23:48:54.867+0000 7fb88c20a640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T23:48:54.855 INFO:tasks.ceph.osd.3.smithi057.stderr:2024-04-25T23:48:54.867+0000 7f3967ac1640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T23:48:55.828 INFO:tasks.ceph.osd.4.smithi080.stderr:2024-04-25T23:48:55.841+0000 7f964c390640 -1 osd.4 0 waiting for initial osdmap 2024-04-25T23:48:55.832 INFO:tasks.ceph.osd.4.smithi080.stderr:2024-04-25T23:48:55.845+0000 7f96441c5640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T23:48:55.843 INFO:tasks.ceph.osd.6.smithi080.stderr:2024-04-25T23:48:55.857+0000 7fa39a145640 -1 osd.6 0 waiting for initial osdmap 2024-04-25T23:48:55.851 INFO:tasks.ceph.osd.6.smithi080.stderr:2024-04-25T23:48:55.865+0000 7fa391f7a640 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T23:48:55.872 INFO:tasks.ceph.osd.5.smithi080.stderr:2024-04-25T23:48:55.885+0000 7f2865d35640 -1 osd.5 0 waiting for initial osdmap 2024-04-25T23:48:55.875 INFO:tasks.ceph.osd.5.smithi080.stderr:2024-04-25T23:48:55.889+0000 7f285d358640 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T23:48:56.888 DEBUG:teuthology.orchestra.run.smithi057:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T23:48:57.013 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.023+0000 7f348c732640 1 Processor -- start 2024-04-25T23:48:57.013 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.023+0000 7f348c732640 1 -- start start 2024-04-25T23:48:57.014 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.027+0000 7f348c732640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3484071270 0x7f34840aec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:57.015 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.027+0000 7f348c732640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34841408f0 con 0x7f348413a010 2024-04-25T23:48:57.015 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.027+0000 7f348c732640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34840734d0 con 0x7f3484071650 2024-04-25T23:48:57.015 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.027+0000 7f348c732640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f34840a94f0 con 0x7f3484071270 2024-04-25T23:48:57.015 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.027+0000 7f348accf640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f348413a010 0x7f348414ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:49574/0 (socket says 172.21.15.57:49574) 2024-04-25T23:48:57.015 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.027+0000 7f348accf640 1 -- 172.21.15.57:0/331551880 learned_addr learned my addr 172.21.15.57:0/331551880 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-04-25T23:48:57.015 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.027+0000 7f348a4ce640 1 --2- 172.21.15.57:0/331551880 >> v2:172.21.15.80:3300/0 conn(0x7f3484071270 0x7f34840aec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:57.015 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.027+0000 7f34894cc640 1 -- 172.21.15.57:0/331551880 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3894608811 0 0) 0x7f347c0014a0 con 0x7f348413a010 2024-04-25T23:48:57.015 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.027+0000 7f34894cc640 1 -- 172.21.15.57:0/331551880 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3470003530 con 0x7f348413a010 2024-04-25T23:48:57.015 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.027+0000 7f34894cc640 1 -- 172.21.15.57:0/331551880 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4218250196 0 0) 0x7f3470003530 con 0x7f348413a010 2024-04-25T23:48:57.015 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.027+0000 7f348a4ce640 1 -- 172.21.15.57:0/331551880 >> v1:172.21.15.80:6789/0 conn(0x7f3484071650 legacy=0x7f3484140db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:57.015 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.027+0000 7f348a4ce640 1 -- 172.21.15.57:0/331551880 >> v1:172.21.15.57:6789/0 conn(0x7f348413a010 legacy=0x7f348414ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:57.015 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.027+0000 7f348a4ce640 1 -- 172.21.15.57:0/331551880 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34840af190 con 0x7f3484071270 2024-04-25T23:48:57.017 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.027+0000 7f348a4ce640 1 --2- 172.21.15.57:0/331551880 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3484071270 0x7f34840aec50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f34780097c0 tx=0x7f347801c310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e883bdcba037bff6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:48:57.017 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.031+0000 7f34894cc640 1 -- 172.21.15.57:0/331551880 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34780b0070 con 0x7f3484071270 2024-04-25T23:48:57.017 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.031+0000 7f34894cc640 1 -- 172.21.15.57:0/331551880 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34780ab400 con 0x7f3484071270 2024-04-25T23:48:57.017 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.031+0000 7f34894cc640 1 -- 172.21.15.57:0/331551880 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f347c001bd0 con 0x7f3484071270 2024-04-25T23:48:57.018 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.031+0000 7f348c732640 1 -- 172.21.15.57:0/331551880 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3484071270 msgr2=0x7f34840aec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:57.018 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.031+0000 7f348c732640 1 --2- 172.21.15.57:0/331551880 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3484071270 0x7f34840aec50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f34780097c0 tx=0x7f347801c310 comp rx=0 tx=0).stop 2024-04-25T23:48:57.018 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.031+0000 7f348c732640 1 -- 172.21.15.57:0/331551880 shutdown_connections 2024-04-25T23:48:57.018 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.031+0000 7f348c732640 1 --2- 172.21.15.57:0/331551880 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3484071270 0x7f34840aec50 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:57.018 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.031+0000 7f348c732640 1 -- 172.21.15.57:0/331551880 >> 172.21.15.57:0/331551880 conn(0x7f34840b4420 msgr2=0x7f34840b4830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:48:57.018 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.031+0000 7f348c732640 1 -- 172.21.15.57:0/331551880 shutdown_connections 2024-04-25T23:48:57.019 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.031+0000 7f348c732640 1 -- 172.21.15.57:0/331551880 wait complete. 2024-04-25T23:48:57.021 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.031+0000 7f348c732640 1 Processor -- start 2024-04-25T23:48:57.021 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.031+0000 7f348c732640 1 -- start start 2024-04-25T23:48:57.021 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.031+0000 7f348c732640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3484071270 0x7f348413fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:57.022 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.031+0000 7f348c732640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3484071650 0x7f348413c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:57.022 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.031+0000 7f348c732640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f348413a010 0x7f348413c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:57.022 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.031+0000 7f348c732640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f348414ee70 con 0x7f3484071650 2024-04-25T23:48:57.022 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.031+0000 7f348c732640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f348413a8b0 con 0x7f3484071270 2024-04-25T23:48:57.022 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.031+0000 7f348c732640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f348414eff0 con 0x7f348413a010 2024-04-25T23:48:57.022 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.031+0000 7f348accf640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f348413a010 0x7f348413c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:57.022 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.031+0000 7f348accf640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f348413a010 0x7f348413c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3301/0 says I am v2:172.21.15.57:47158/0 (socket says 172.21.15.57:47158) 2024-04-25T23:48:57.022 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.031+0000 7f348accf640 1 -- 172.21.15.57:0/2767891268 learned_addr learned my addr 172.21.15.57:0/2767891268 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:48:57.022 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.031+0000 7f3489ccd640 1 --2- 172.21.15.57:0/2767891268 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3484071650 0x7f348413c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:57.022 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.031+0000 7f348a4ce640 1 --2- 172.21.15.57:0/2767891268 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3484071270 0x7f348413fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:57.022 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.031+0000 7f348accf640 1 -- 172.21.15.57:0/2767891268 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3484071270 msgr2=0x7f348413fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:57.023 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.031+0000 7f348accf640 1 --2- 172.21.15.57:0/2767891268 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3484071270 0x7f348413fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:57.023 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.031+0000 7f348accf640 1 -- 172.21.15.57:0/2767891268 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3484071650 msgr2=0x7f348413c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:57.023 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.031+0000 7f348accf640 1 --2- 172.21.15.57:0/2767891268 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3484071650 0x7f348413c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:57.023 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.031+0000 7f348accf640 1 -- 172.21.15.57:0/2767891268 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f348413b5e0 con 0x7f348413a010 2024-04-25T23:48:57.023 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.031+0000 7f348a4ce640 1 --2- 172.21.15.57:0/2767891268 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3484071270 0x7f348413fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T23:48:57.023 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.031+0000 7f348accf640 1 --2- 172.21.15.57:0/2767891268 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f348413a010 0x7f348413c840 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f347c00cb90 tx=0x7f347c00a430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:48:57.023 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.031+0000 7f3476ffd640 1 -- 172.21.15.57:0/2767891268 <== mon.2 v2:172.21.15.57:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f347c017070 con 0x7f348413a010 2024-04-25T23:48:57.023 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.031+0000 7f3476ffd640 1 -- 172.21.15.57:0/2767891268 <== mon.2 v2:172.21.15.57:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f347c00ad40 con 0x7f348413a010 2024-04-25T23:48:57.023 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.031+0000 7f348c732640 1 -- 172.21.15.57:0/2767891268 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f348413f050 con 0x7f348413a010 2024-04-25T23:48:57.023 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.031+0000 7f3476ffd640 1 -- 172.21.15.57:0/2767891268 <== mon.2 v2:172.21.15.57:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f347c012640 con 0x7f348413a010 2024-04-25T23:48:57.023 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.031+0000 7f348c732640 1 -- 172.21.15.57:0/2767891268 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3484140190 con 0x7f348413a010 2024-04-25T23:48:57.024 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.035+0000 7f348c732640 1 -- 172.21.15.57:0/2767891268 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f348404f510 con 0x7f348413a010 2024-04-25T23:48:57.024 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.035+0000 7f3476ffd640 1 -- 172.21.15.57:0/2767891268 <== mon.2 v2:172.21.15.57:3301/0 4 ==== mgrmap(e 4) ==== 87354+0+0 (secure 0 0 0) 0x7f347c01a070 con 0x7f348413a010 2024-04-25T23:48:57.024 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.035+0000 7f3476ffd640 1 --2- 172.21.15.57:0/2767891268 >> v2:172.21.15.80:6816/2493457677 conn(0x7f3468067820 0x7f3468069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:57.025 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.035+0000 7f3476ffd640 1 -- 172.21.15.57:0/2767891268 <== mon.2 v2:172.21.15.57:3301/0 5 ==== osd_map(14..14 src has 1..14) ==== 3351+0+0 (secure 0 0 0) 0x7f347c084f90 con 0x7f348413a010 2024-04-25T23:48:57.025 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.035+0000 7f348a4ce640 1 --2- 172.21.15.57:0/2767891268 >> v2:172.21.15.80:6816/2493457677 conn(0x7f3468067820 0x7f3468069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:57.025 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.035+0000 7f348a4ce640 1 --2- 172.21.15.57:0/2767891268 >> v2:172.21.15.80:6816/2493457677 conn(0x7f3468067820 0x7f3468069ce0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f34780ad000 tx=0x7f34780a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:48:57.027 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.039+0000 7f3476ffd640 1 -- 172.21.15.57:0/2767891268 <== mon.2 v2:172.21.15.57:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f34841408f0 con 0x7f348413a010 2024-04-25T23:48:57.190 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.203+0000 7f348c732640 1 -- 172.21.15.57:0/2767891268 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f348406f090 con 0x7f348413a010 2024-04-25T23:48:57.191 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.203+0000 7f3476ffd640 1 -- 172.21.15.57:0/2767891268 <== mon.2 v2:172.21.15.57:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) ==== 74+0+9060 (secure 0 0 0) 0x7f347c055f20 con 0x7f348413a010 2024-04-25T23:48:57.191 INFO:teuthology.misc.health.smithi057.stdout: 2024-04-25T23:48:57.192 INFO:teuthology.misc.health.smithi057.stdout:{"epoch":14,"fsid":"ca0adda3-98dd-4ba5-b4b2-8b4f20dcabdd","created":"2024-04-25T23:48:46.211894+0000","modified":"2024-04-25T23:48:56.859933+0000","last_up_change":"2024-04-25T23:48:56.859933+0000","last_in_change":"2024-04-25T23:48:48.838840+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"5986c317-8828-4568-af68-a8ae96690c51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":2528545516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6813","nonce":2528545516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6815","nonce":2528545516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":2528545516}]},"public_addr":"172.21.15.57:6812/2528545516","cluster_addr":"172.21.15.57:6813/2528545516","heartbeat_back_addr":"172.21.15.57:6815/2528545516","heartbeat_front_addr":"172.21.15.57:6814/2528545516","state":["exists","up"]},{"osd":1,"uuid":"1aa157dd-52fe-4bb3-9f63-4db493de5333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":2225240106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6809","nonce":2225240106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6811","nonce":2225240106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":2225240106}]},"public_addr":"172.21.15.57:6808/2225240106","cluster_addr":"172.21.15.57:6809/2225240106","heartbeat_back_addr":"172.21.15.57:6811/2225240106","heartbeat_front_addr":"172.21.15.57:6810/2225240106","state":["exists","up"]},{"osd":2,"uuid":"1391be38-25ac-43f0-8090-15adddc6d69d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":1976583347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6805","nonce":1976583347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6807","nonce":1976583347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":1976583347}]},"public_addr":"172.21.15.57:6804/1976583347","cluster_addr":"172.21.15.57:6805/1976583347","heartbeat_back_addr":"172.21.15.57:6807/1976583347","heartbeat_front_addr":"172.21.15.57:6806/1976583347","state":["exists","up"]},{"osd":3,"uuid":"a85e7650-effb-4ea7-9f3b-9fc9fc1576e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":1899803014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6801","nonce":1899803014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6803","nonce":1899803014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":1899803014}]},"public_addr":"172.21.15.57:6800/1899803014","cluster_addr":"172.21.15.57:6801/1899803014","heartbeat_back_addr":"172.21.15.57:6803/1899803014","heartbeat_front_addr":"172.21.15.57:6802/1899803014","state":["exists","up"]},{"osd":4,"uuid":"ad3fb1a9-cec6-4fb3-9f49-22b9480f524b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":4069821322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":4069821322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":4069821322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":4069821322}]},"public_addr":"172.21.15.80:6800/4069821322","cluster_addr":"172.21.15.80:6802/4069821322","heartbeat_back_addr":"172.21.15.80:6806/4069821322","heartbeat_front_addr":"172.21.15.80:6804/4069821322","state":["exists","up"]},{"osd":5,"uuid":"fd05cda6-27a5-477a-8916-b34b69ba9ef6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6811","nonce":2954555099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":2954555099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6815","nonce":2954555099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2954555099}]},"public_addr":"172.21.15.80:6811/2954555099","cluster_addr":"172.21.15.80:6813/2954555099","heartbeat_back_addr":"172.21.15.80:6815/2954555099","heartbeat_front_addr":"172.21.15.80:6814/2954555099","state":["exists","up"]},{"osd":6,"uuid":"599ef421-483d-4225-a98f-7405006b1809","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6801","nonce":163896711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6803","nonce":163896711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6807","nonce":163896711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6805","nonce":163896711}]},"public_addr":"172.21.15.80:6801/163896711","cluster_addr":"172.21.15.80:6803/163896711","heartbeat_back_addr":"172.21.15.80:6807/163896711","heartbeat_front_addr":"172.21.15.80:6805/163896711","state":["exists","up"]},{"osd":7,"uuid":"0e9d533c-465f-403c-8fe8-7dafc953d774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2647667231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":2647667231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2647667231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2647667231}]},"public_addr":"172.21.15.80:6808/2647667231","cluster_addr":"172.21.15.80:6809/2647667231","heartbeat_back_addr":"172.21.15.80:6812/2647667231","heartbeat_front_addr":"172.21.15.80:6810/2647667231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T23:48:54.460484+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T23:48:54.420671+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T23:48:54.335197+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T23:48:54.243121+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T23:48:54.129310+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T23:48:57.193 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.207+0000 7f348c732640 1 -- 172.21.15.57:0/2767891268 >> v2:172.21.15.80:6816/2493457677 conn(0x7f3468067820 msgr2=0x7f3468069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:57.194 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.207+0000 7f348c732640 1 --2- 172.21.15.57:0/2767891268 >> v2:172.21.15.80:6816/2493457677 conn(0x7f3468067820 0x7f3468069ce0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f34780ad000 tx=0x7f34780a6000 comp rx=0 tx=0).stop 2024-04-25T23:48:57.194 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.207+0000 7f348c732640 1 -- 172.21.15.57:0/2767891268 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f348413a010 msgr2=0x7f348413c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:57.194 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.207+0000 7f348c732640 1 --2- 172.21.15.57:0/2767891268 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f348413a010 0x7f348413c840 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f347c00cb90 tx=0x7f347c00a430 comp rx=0 tx=0).stop 2024-04-25T23:48:57.194 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.207+0000 7f348c732640 1 -- 172.21.15.57:0/2767891268 shutdown_connections 2024-04-25T23:48:57.194 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.207+0000 7f348c732640 1 --2- 172.21.15.57:0/2767891268 >> v2:172.21.15.80:6816/2493457677 conn(0x7f3468067820 0x7f3468069ce0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:57.194 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.207+0000 7f348c732640 1 --2- 172.21.15.57:0/2767891268 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f348413a010 0x7f348413c840 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:57.194 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.207+0000 7f348c732640 1 --2- 172.21.15.57:0/2767891268 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3484071650 0x7f348413c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:57.194 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.207+0000 7f348c732640 1 --2- 172.21.15.57:0/2767891268 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3484071270 0x7f348413fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:57.194 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.207+0000 7f348c732640 1 -- 172.21.15.57:0/2767891268 >> 172.21.15.57:0/2767891268 conn(0x7f34840b4420 msgr2=0x7f34840afab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:48:57.194 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.207+0000 7f348c732640 1 -- 172.21.15.57:0/2767891268 shutdown_connections 2024-04-25T23:48:57.195 INFO:teuthology.misc.health.smithi057.stderr:2024-04-25T23:48:57.207+0000 7f348c732640 1 -- 172.21.15.57:0/2767891268 wait complete. 2024-04-25T23:48:57.210 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-25T23:48:57.210 INFO:tasks.ceph:Creating RBD pool 2024-04-25T23:48:57.210 DEBUG:teuthology.orchestra.run.smithi057:> sudo ceph --cluster ceph osd pool create rbd 8 2024-04-25T23:48:57.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.355+0000 7fe17619f640 1 Processor -- start 2024-04-25T23:48:57.341 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.355+0000 7fe17619f640 1 -- start start 2024-04-25T23:48:57.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.355+0000 7fe17619f640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fe170071470 0x7fe1700ad9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:57.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.355+0000 7fe17619f640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fe170072bc0 0x7fe170143b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:57.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.355+0000 7fe17619f640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe170073110 con 0x7fe1701440d0 2024-04-25T23:48:57.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.355+0000 7fe17619f640 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 -- 0x7fe1700b24b0 con 0x7fe170071470 2024-04-25T23:48:57.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.355+0000 7fe17619f640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7fe1700733f0 con 0x7fe170072bc0 2024-04-25T23:48:57.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.355+0000 7fe167fff640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fe170072bc0 0x7fe170143b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:57.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.355+0000 7fe16f7fe640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fe170071470 0x7fe1700ad9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:57.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.355+0000 7fe16f7fe640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fe170071470 0x7fe1700ad9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33228/0 (socket says 172.21.15.57:33228) 2024-04-25T23:48:57.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.355+0000 7fe167fff640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fe170072bc0 0x7fe170143b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3301/0 says I am v2:172.21.15.57:47164/0 (socket says 172.21.15.57:47164) 2024-04-25T23:48:57.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.355+0000 7fe16f7fe640 1 -- 172.21.15.57:0/2290499134 learned_addr learned my addr 172.21.15.57:0/2290499134 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:48:57.343 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.355+0000 7fe16f7fe640 1 -- 172.21.15.57:0/2290499134 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fe170072bc0 msgr2=0x7fe170143b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:48:57.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.355+0000 7fe16f7fe640 1 --2- 172.21.15.57:0/2290499134 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fe170072bc0 0x7fe170143b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:57.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.355+0000 7fe16f7fe640 1 -- 172.21.15.57:0/2290499134 >> v1:172.21.15.80:6789/0 conn(0x7fe1701440d0 legacy=0x7fe1700b0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:57.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.355+0000 7fe16f7fe640 1 -- 172.21.15.57:0/2290499134 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1700ad5d0 con 0x7fe170071470 2024-04-25T23:48:57.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.355+0000 7fe16f7fe640 1 --2- 172.21.15.57:0/2290499134 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe170071470 0x7fe1700ad9f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe160009870 tx=0x7fe16001c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1849c516452dd692 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:48:57.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.359+0000 7fe16effd640 1 -- 172.21.15.57:0/2290499134 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1600b1070 con 0x7fe170071470 2024-04-25T23:48:57.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.359+0000 7fe16effd640 1 -- 172.21.15.57:0/2290499134 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1600ac620 con 0x7fe170071470 2024-04-25T23:48:57.345 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.359+0000 7fe16effd640 1 -- 172.21.15.57:0/2290499134 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1600ac9f0 con 0x7fe170071470 2024-04-25T23:48:57.346 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.359+0000 7fe17619f640 1 -- 172.21.15.57:0/2290499134 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe170071470 msgr2=0x7fe1700ad9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:57.346 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.359+0000 7fe17619f640 1 --2- 172.21.15.57:0/2290499134 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe170071470 0x7fe1700ad9f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe160009870 tx=0x7fe16001c4b0 comp rx=0 tx=0).stop 2024-04-25T23:48:57.346 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.359+0000 7fe17619f640 1 -- 172.21.15.57:0/2290499134 shutdown_connections 2024-04-25T23:48:57.346 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.359+0000 7fe17619f640 1 --2- 172.21.15.57:0/2290499134 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fe170072bc0 0x7fe170143b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:57.346 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.359+0000 7fe17619f640 1 --2- 172.21.15.57:0/2290499134 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe170071470 0x7fe1700ad9f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:57.346 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.359+0000 7fe17619f640 1 -- 172.21.15.57:0/2290499134 >> 172.21.15.57:0/2290499134 conn(0x7fe1700b6200 msgr2=0x7fe1700b6610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:48:57.346 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.359+0000 7fe17619f640 1 -- 172.21.15.57:0/2290499134 shutdown_connections 2024-04-25T23:48:57.346 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.359+0000 7fe17619f640 1 -- 172.21.15.57:0/2290499134 wait complete. 2024-04-25T23:48:57.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.359+0000 7fe17619f640 1 Processor -- start 2024-04-25T23:48:57.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.359+0000 7fe17619f640 1 -- start start 2024-04-25T23:48:57.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.359+0000 7fe17619f640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe170071470 0x7fe1700aa9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:57.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.359+0000 7fe17619f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe170072bc0 0x7fe1700b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:57.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.359+0000 7fe17619f640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fe1701440d0 0x7fe1700ab540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:57.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.359+0000 7fe17619f640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fe17014dc70 con 0x7fe170071470 2024-04-25T23:48:57.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.359+0000 7fe17619f640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe17014daf0 con 0x7fe170072bc0 2024-04-25T23:48:57.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.359+0000 7fe17619f640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7fe17014ddf0 con 0x7fe1701440d0 2024-04-25T23:48:57.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.359+0000 7fe16ffff640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fe1701440d0 0x7fe1700ab540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:57.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.359+0000 7fe16ffff640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fe1701440d0 0x7fe1700ab540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3301/0 says I am v2:172.21.15.57:47166/0 (socket says 172.21.15.57:47166) 2024-04-25T23:48:57.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.359+0000 7fe16ffff640 1 -- 172.21.15.57:0/3960555423 learned_addr learned my addr 172.21.15.57:0/3960555423 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:48:57.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.359+0000 7fe16f7fe640 1 --2- 172.21.15.57:0/3960555423 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe170071470 0x7fe1700aa9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:57.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.359+0000 7fe167fff640 1 --2- 172.21.15.57:0/3960555423 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe170072bc0 0x7fe1700b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:57.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.359+0000 7fe16ffff640 1 -- 172.21.15.57:0/3960555423 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe170072bc0 msgr2=0x7fe1700b4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:57.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.359+0000 7fe16ffff640 1 --2- 172.21.15.57:0/3960555423 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe170072bc0 0x7fe1700b4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:57.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.359+0000 7fe16ffff640 1 -- 172.21.15.57:0/3960555423 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe170071470 msgr2=0x7fe1700aa9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:57.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.359+0000 7fe16ffff640 1 --2- 172.21.15.57:0/3960555423 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe170071470 0x7fe1700aa9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:57.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.359+0000 7fe16ffff640 1 -- 172.21.15.57:0/3960555423 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1701457c0 con 0x7fe1701440d0 2024-04-25T23:48:57.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.359+0000 7fe16f7fe640 1 --2- 172.21.15.57:0/3960555423 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe170071470 0x7fe1700aa9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T23:48:57.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.359+0000 7fe16ffff640 1 --2- 172.21.15.57:0/3960555423 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fe1701440d0 0x7fe1700ab540 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe16800c760 tx=0x7fe16800cb60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:48:57.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.359+0000 7fe16cff9640 1 -- 172.21.15.57:0/3960555423 <== mon.2 v2:172.21.15.57:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe168016020 con 0x7fe1701440d0 2024-04-25T23:48:57.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.359+0000 7fe16cff9640 1 -- 172.21.15.57:0/3960555423 <== mon.2 v2:172.21.15.57:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe168006d60 con 0x7fe1701440d0 2024-04-25T23:48:57.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.359+0000 7fe17619f640 1 -- 172.21.15.57:0/3960555423 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1700aafd0 con 0x7fe1701440d0 2024-04-25T23:48:57.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.359+0000 7fe16cff9640 1 -- 172.21.15.57:0/3960555423 <== mon.2 v2:172.21.15.57:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe168003bc0 con 0x7fe1701440d0 2024-04-25T23:48:57.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.359+0000 7fe17619f640 1 -- 172.21.15.57:0/3960555423 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1700ac5c0 con 0x7fe1701440d0 2024-04-25T23:48:57.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.363+0000 7fe17619f640 1 -- 172.21.15.57:0/3960555423 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe170068a10 con 0x7fe1701440d0 2024-04-25T23:48:57.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.363+0000 7fe16cff9640 1 -- 172.21.15.57:0/3960555423 <== mon.2 v2:172.21.15.57:3301/0 4 ==== mgrmap(e 4) ==== 87354+0+0 (secure 0 0 0) 0x7fe1680050d0 con 0x7fe1701440d0 2024-04-25T23:48:57.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.363+0000 7fe16cff9640 1 --2- 172.21.15.57:0/3960555423 >> v2:172.21.15.80:6816/2493457677 conn(0x7fe1500677f0 0x7fe150069cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:57.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.363+0000 7fe16cff9640 1 -- 172.21.15.57:0/3960555423 <== mon.2 v2:172.21.15.57:3301/0 5 ==== osd_map(14..14 src has 1..14) ==== 3351+0+0 (secure 0 0 0) 0x7fe168084da0 con 0x7fe1701440d0 2024-04-25T23:48:57.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.363+0000 7fe16f7fe640 1 --2- 172.21.15.57:0/3960555423 >> v2:172.21.15.80:6816/2493457677 conn(0x7fe1500677f0 0x7fe150069cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:57.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.363+0000 7fe16f7fe640 1 --2- 172.21.15.57:0/3960555423 >> v2:172.21.15.80:6816/2493457677 conn(0x7fe1500677f0 0x7fe150069cb0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe1600ae000 tx=0x7fe160008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:48:57.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.367+0000 7fe16cff9640 1 -- 172.21.15.57:0/3960555423 <== mon.2 v2:172.21.15.57:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe168020020 con 0x7fe1701440d0 2024-04-25T23:48:57.520 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.531+0000 7fe17619f640 1 -- 172.21.15.57:0/3960555423 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) -- 0x7fe17006d980 con 0x7fe1701440d0 2024-04-25T23:48:57.869 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.879+0000 7fe16cff9640 1 -- 172.21.15.57:0/3960555423 <== mon.2 v2:172.21.15.57:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v15) ==== 109+0+0 (secure 0 0 0) 0x7fe1680562e0 con 0x7fe1701440d0 2024-04-25T23:48:57.869 INFO:teuthology.orchestra.run.smithi057.stderr:pool 'rbd' created 2024-04-25T23:48:57.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.887+0000 7fe17619f640 1 -- 172.21.15.57:0/3960555423 >> v2:172.21.15.80:6816/2493457677 conn(0x7fe1500677f0 msgr2=0x7fe150069cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:57.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.887+0000 7fe17619f640 1 --2- 172.21.15.57:0/3960555423 >> v2:172.21.15.80:6816/2493457677 conn(0x7fe1500677f0 0x7fe150069cb0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe1600ae000 tx=0x7fe160008040 comp rx=0 tx=0).stop 2024-04-25T23:48:57.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.887+0000 7fe17619f640 1 -- 172.21.15.57:0/3960555423 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fe1701440d0 msgr2=0x7fe1700ab540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:57.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.887+0000 7fe17619f640 1 --2- 172.21.15.57:0/3960555423 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fe1701440d0 0x7fe1700ab540 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe16800c760 tx=0x7fe16800cb60 comp rx=0 tx=0).stop 2024-04-25T23:48:57.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.887+0000 7fe17619f640 1 -- 172.21.15.57:0/3960555423 shutdown_connections 2024-04-25T23:48:57.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.887+0000 7fe17619f640 1 --2- 172.21.15.57:0/3960555423 >> v2:172.21.15.80:6816/2493457677 conn(0x7fe1500677f0 0x7fe150069cb0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:57.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.887+0000 7fe17619f640 1 --2- 172.21.15.57:0/3960555423 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe170072bc0 0x7fe1700b4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:57.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.887+0000 7fe17619f640 1 --2- 172.21.15.57:0/3960555423 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fe1701440d0 0x7fe1700ab540 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:57.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.887+0000 7fe17619f640 1 --2- 172.21.15.57:0/3960555423 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe170071470 0x7fe1700aa9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:57.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.887+0000 7fe17619f640 1 -- 172.21.15.57:0/3960555423 >> 172.21.15.57:0/3960555423 conn(0x7fe1700b6200 msgr2=0x7fe1701419e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:48:57.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.887+0000 7fe17619f640 1 -- 172.21.15.57:0/3960555423 shutdown_connections 2024-04-25T23:48:57.876 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:57.887+0000 7fe17619f640 1 -- 172.21.15.57:0/3960555423 wait complete. 2024-04-25T23:48:57.899 DEBUG:teuthology.orchestra.run.smithi057:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2024-04-25T23:48:58.032 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.043+0000 7f114dca5640 1 Processor -- start 2024-04-25T23:48:58.032 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.043+0000 7f114dca5640 1 -- start start 2024-04-25T23:48:58.033 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.043+0000 7f114dca5640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1148140ec0 0x7f11480adf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:58.033 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.043+0000 7f114dca5640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f11480ae4a0 0x7f11480aebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:58.033 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.043+0000 7f114dca5640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1148141350 con 0x7f1148070eb0 2024-04-25T23:48:58.033 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.043+0000 7f114dca5640 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 -- 0x7f1148072e40 con 0x7f11480ae4a0 2024-04-25T23:48:58.033 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.043+0000 7f114dca5640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f1148073090 con 0x7f1148140ec0 2024-04-25T23:48:58.033 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.043+0000 7f1147fff640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f11480ae4a0 0x7f11480aebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:58.033 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.043+0000 7f1147fff640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f11480ae4a0 0x7f11480aebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33270/0 (socket says 172.21.15.57:33270) 2024-04-25T23:48:58.033 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.043+0000 7f1147fff640 1 -- 172.21.15.57:0/908874786 learned_addr learned my addr 172.21.15.57:0/908874786 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:48:58.034 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.043+0000 7f1146ffd640 1 --2- 172.21.15.57:0/908874786 >> v2:172.21.15.80:3300/0 conn(0x7f1148140ec0 0x7f11480adf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:58.034 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.047+0000 7f1147fff640 1 -- 172.21.15.57:0/908874786 >> v2:172.21.15.80:3300/0 conn(0x7f1148140ec0 msgr2=0x7f11480adf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:48:58.034 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.047+0000 7f1147fff640 1 --2- 172.21.15.57:0/908874786 >> v2:172.21.15.80:3300/0 conn(0x7f1148140ec0 0x7f11480adf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:58.034 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.047+0000 7f1147fff640 1 -- 172.21.15.57:0/908874786 >> v1:172.21.15.57:6789/0 conn(0x7f1148070eb0 legacy=0x7f1148071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:58.034 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.047+0000 7f1147fff640 1 -- 172.21.15.57:0/908874786 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1148073370 con 0x7f11480ae4a0 2024-04-25T23:48:58.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.047+0000 7f1147fff640 1 --2- 172.21.15.57:0/908874786 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11480ae4a0 0x7f11480aebb0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f114813e1c0 tx=0x7f114001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=decf75b19135907e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:48:58.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.047+0000 7f11467fc640 1 -- 172.21.15.57:0/908874786 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11400a5860 con 0x7f11480ae4a0 2024-04-25T23:48:58.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.047+0000 7f11467fc640 1 -- 172.21.15.57:0/908874786 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11400a5a00 con 0x7f11480ae4a0 2024-04-25T23:48:58.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.047+0000 7f11467fc640 1 -- 172.21.15.57:0/908874786 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11400a5dd0 con 0x7f11480ae4a0 2024-04-25T23:48:58.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.047+0000 7f114dca5640 1 -- 172.21.15.57:0/908874786 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11480ae4a0 msgr2=0x7f11480aebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:58.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.047+0000 7f114dca5640 1 --2- 172.21.15.57:0/908874786 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11480ae4a0 0x7f11480aebb0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f114813e1c0 tx=0x7f114001c9a0 comp rx=0 tx=0).stop 2024-04-25T23:48:58.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.047+0000 7f114dca5640 1 -- 172.21.15.57:0/908874786 shutdown_connections 2024-04-25T23:48:58.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.047+0000 7f114dca5640 1 --2- 172.21.15.57:0/908874786 >> v2:172.21.15.80:3300/0 conn(0x7f1148140ec0 0x7f11480adf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:58.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.047+0000 7f114dca5640 1 --2- 172.21.15.57:0/908874786 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11480ae4a0 0x7f11480aebb0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:58.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.047+0000 7f114dca5640 1 -- 172.21.15.57:0/908874786 >> 172.21.15.57:0/908874786 conn(0x7f11480b4380 msgr2=0x7f11480b4790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:48:58.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.047+0000 7f114dca5640 1 -- 172.21.15.57:0/908874786 shutdown_connections 2024-04-25T23:48:58.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.047+0000 7f114dca5640 1 -- 172.21.15.57:0/908874786 wait complete. 2024-04-25T23:48:58.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.051+0000 7f114dca5640 1 Processor -- start 2024-04-25T23:48:58.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.051+0000 7f114dca5640 1 -- start start 2024-04-25T23:48:58.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.051+0000 7f114dca5640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1148070eb0 0x7f11481440b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:58.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.051+0000 7f114dca5640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f11480ae4a0 0x7f11481d90b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:58.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.051+0000 7f114dca5640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f1148140ec0 0x7f11481d95f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:58.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.051+0000 7f114dca5640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f11480afd10 con 0x7f1148070eb0 2024-04-25T23:48:58.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.051+0000 7f114dca5640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f11480afb90 con 0x7f11480ae4a0 2024-04-25T23:48:58.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.051+0000 7f114dca5640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f11480afe90 con 0x7f1148140ec0 2024-04-25T23:48:58.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.051+0000 7f11477fe640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1148070eb0 0x7f11481440b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:58.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.051+0000 7f11477fe640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1148070eb0 0x7f11481440b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33286/0 (socket says 172.21.15.57:33286) 2024-04-25T23:48:58.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.051+0000 7f1147fff640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f1148140ec0 0x7f11481d95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:58.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.051+0000 7f11477fe640 1 -- 172.21.15.57:0/4148932748 learned_addr learned my addr 172.21.15.57:0/4148932748 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:48:58.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.051+0000 7f1146ffd640 1 --2- 172.21.15.57:0/4148932748 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f11480ae4a0 0x7f11481d90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:58.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.051+0000 7f11477fe640 1 -- 172.21.15.57:0/4148932748 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f1148140ec0 msgr2=0x7f11481d95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:58.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.051+0000 7f11477fe640 1 --2- 172.21.15.57:0/4148932748 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f1148140ec0 0x7f11481d95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:58.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.051+0000 7f11477fe640 1 -- 172.21.15.57:0/4148932748 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f11480ae4a0 msgr2=0x7f11481d90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:58.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.051+0000 7f11477fe640 1 --2- 172.21.15.57:0/4148932748 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f11480ae4a0 0x7f11481d90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:58.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.051+0000 7f11477fe640 1 -- 172.21.15.57:0/4148932748 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1148143650 con 0x7f1148070eb0 2024-04-25T23:48:58.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.051+0000 7f1146ffd640 1 --2- 172.21.15.57:0/4148932748 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f11480ae4a0 0x7f11481d90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T23:48:58.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.051+0000 7f1147fff640 1 --2- 172.21.15.57:0/4148932748 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f1148140ec0 0x7f11481d95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T23:48:58.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.051+0000 7f11477fe640 1 --2- 172.21.15.57:0/4148932748 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1148070eb0 0x7f11481440b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f113800c420 tx=0x7f113800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:48:58.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.051+0000 7f112ffff640 1 -- 172.21.15.57:0/4148932748 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11380052d0 con 0x7f1148070eb0 2024-04-25T23:48:58.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.051+0000 7f112ffff640 1 -- 172.21.15.57:0/4148932748 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1138005470 con 0x7f1148070eb0 2024-04-25T23:48:58.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.051+0000 7f114dca5640 1 -- 172.21.15.57:0/4148932748 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1148143930 con 0x7f1148070eb0 2024-04-25T23:48:58.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.051+0000 7f112ffff640 1 -- 172.21.15.57:0/4148932748 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1138007260 con 0x7f1148070eb0 2024-04-25T23:48:58.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.051+0000 7f114dca5640 1 -- 172.21.15.57:0/4148932748 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f114813a0f0 con 0x7f1148070eb0 2024-04-25T23:48:58.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.051+0000 7f114dca5640 1 -- 172.21.15.57:0/4148932748 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11481d8d00 con 0x7f1148070eb0 2024-04-25T23:48:58.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.051+0000 7f112ffff640 1 -- 172.21.15.57:0/4148932748 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 4) ==== 87354+0+0 (secure 0 0 0) 0x7f1138007400 con 0x7f1148070eb0 2024-04-25T23:48:58.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.055+0000 7f112ffff640 1 --2- 172.21.15.57:0/4148932748 >> v2:172.21.15.80:6816/2493457677 conn(0x7f11340677f0 0x7f1134069cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:58.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.055+0000 7f112ffff640 1 -- 172.21.15.57:0/4148932748 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 4068+0+0 (secure 0 0 0) 0x7f11380848e0 con 0x7f1148070eb0 2024-04-25T23:48:58.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.055+0000 7f1146ffd640 1 --2- 172.21.15.57:0/4148932748 >> v2:172.21.15.80:6816/2493457677 conn(0x7f11340677f0 0x7f1134069cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:58.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.055+0000 7f1146ffd640 1 --2- 172.21.15.57:0/4148932748 >> v2:172.21.15.80:6816/2493457677 conn(0x7f11340677f0 0x7f1134069cb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f114813e140 tx=0x7f113c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:48:58.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.059+0000 7f112ffff640 1 -- 172.21.15.57:0/4148932748 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f11481d8d00 con 0x7f1148070eb0 2024-04-25T23:48:58.213 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.227+0000 7f114dca5640 1 -- 172.21.15.57:0/4148932748 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) -- 0x7f1148068a10 con 0x7f1148070eb0 2024-04-25T23:48:58.874 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.887+0000 7f112ffff640 1 -- 172.21.15.57:0/4148932748 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v16) ==== 173+0+0 (secure 0 0 0) 0x7f11380555a0 con 0x7f1148070eb0 2024-04-25T23:48:58.874 INFO:teuthology.orchestra.run.smithi057.stderr:enabled application 'rbd' on pool 'rbd' 2024-04-25T23:48:58.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.891+0000 7f114dca5640 1 -- 172.21.15.57:0/4148932748 >> v2:172.21.15.80:6816/2493457677 conn(0x7f11340677f0 msgr2=0x7f1134069cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:58.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.891+0000 7f114dca5640 1 --2- 172.21.15.57:0/4148932748 >> v2:172.21.15.80:6816/2493457677 conn(0x7f11340677f0 0x7f1134069cb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f114813e140 tx=0x7f113c008040 comp rx=0 tx=0).stop 2024-04-25T23:48:58.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.891+0000 7f114dca5640 1 -- 172.21.15.57:0/4148932748 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1148070eb0 msgr2=0x7f11481440b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:58.879 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.891+0000 7f114dca5640 1 --2- 172.21.15.57:0/4148932748 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1148070eb0 0x7f11481440b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f113800c420 tx=0x7f113800c8e0 comp rx=0 tx=0).stop 2024-04-25T23:48:58.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.891+0000 7f114dca5640 1 -- 172.21.15.57:0/4148932748 shutdown_connections 2024-04-25T23:48:58.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.891+0000 7f114dca5640 1 --2- 172.21.15.57:0/4148932748 >> v2:172.21.15.80:6816/2493457677 conn(0x7f11340677f0 0x7f1134069cb0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:58.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.891+0000 7f114dca5640 1 --2- 172.21.15.57:0/4148932748 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f11480ae4a0 0x7f11481d90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:58.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.891+0000 7f114dca5640 1 --2- 172.21.15.57:0/4148932748 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f1148140ec0 0x7f11481d95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:58.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.891+0000 7f114dca5640 1 --2- 172.21.15.57:0/4148932748 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1148070eb0 0x7f11481440b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:58.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.891+0000 7f114dca5640 1 -- 172.21.15.57:0/4148932748 >> 172.21.15.57:0/4148932748 conn(0x7f11480b4380 msgr2=0x7f11481328f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:48:58.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.891+0000 7f114dca5640 1 -- 172.21.15.57:0/4148932748 shutdown_connections 2024-04-25T23:48:58.880 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:58.891+0000 7f114dca5640 1 -- 172.21.15.57:0/4148932748 wait complete. 2024-04-25T23:48:58.907 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-25T23:48:58.907 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T23:48:58.908 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-25T23:48:58.908 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-25T23:48:58.908 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-25T23:48:59.034 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.047+0000 7f27f68b1640 1 Processor -- start 2024-04-25T23:48:59.034 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.047+0000 7f27f68b1640 1 -- start start 2024-04-25T23:48:59.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.047+0000 7f27f68b1640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f27f00b1670 0x7f27f0140340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:59.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.047+0000 7f27f68b1640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27f00a9570 con 0x7f27f006f490 2024-04-25T23:48:59.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.047+0000 7f27f68b1640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27f00723e0 con 0x7f27f00b1290 2024-04-25T23:48:59.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.047+0000 7f27f68b1640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f27f0073540 con 0x7f27f00b1670 2024-04-25T23:48:59.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.047+0000 7f27f4e4e640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f27f006f490 0x7f27f014ac50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:49584/0 (socket says 172.21.15.57:49584) 2024-04-25T23:48:59.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.047+0000 7f27f4e4e640 1 -- 172.21.15.57:0/2742745664 learned_addr learned my addr 172.21.15.57:0/2742745664 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-04-25T23:48:59.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.047+0000 7f27ef7fe640 1 --2- 172.21.15.57:0/2742745664 >> v2:172.21.15.80:3300/0 conn(0x7f27f00b1670 0x7f27f0140340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:59.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.047+0000 7f27eeffd640 1 -- 172.21.15.57:0/2742745664 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3900022741 0 0) 0x7f27f00a9570 con 0x7f27f006f490 2024-04-25T23:48:59.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.047+0000 7f27eeffd640 1 -- 172.21.15.57:0/2742745664 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f27d8003530 con 0x7f27f006f490 2024-04-25T23:48:59.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.047+0000 7f27eeffd640 1 -- 172.21.15.57:0/2742745664 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 858811812 0 0) 0x7f27d8003530 con 0x7f27f006f490 2024-04-25T23:48:59.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.047+0000 7f27eeffd640 1 -- 172.21.15.57:0/2742745664 >> v2:172.21.15.80:3300/0 conn(0x7f27f00b1670 msgr2=0x7f27f0140340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:48:59.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.047+0000 7f27eeffd640 1 --2- 172.21.15.57:0/2742745664 >> v2:172.21.15.80:3300/0 conn(0x7f27f00b1670 0x7f27f0140340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:59.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.047+0000 7f27eeffd640 1 -- 172.21.15.57:0/2742745664 >> v1:172.21.15.80:6789/0 conn(0x7f27f00b1290 legacy=0x7f27f0139e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:59.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.047+0000 7f27eeffd640 1 -- 172.21.15.57:0/2742745664 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27f0140f90 con 0x7f27f006f490 2024-04-25T23:48:59.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.047+0000 7f27eeffd640 1 -- 172.21.15.57:0/2742745664 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2627952221 0 0) 0x7f27e8001a00 con 0x7f27f006f490 2024-04-25T23:48:59.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.047+0000 7f27eeffd640 1 -- 172.21.15.57:0/2742745664 >> v1:172.21.15.57:6789/0 conn(0x7f27f006f490 legacy=0x7f27f014ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:59.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.047+0000 7f27eeffd640 1 --2- 172.21.15.57:0/2742745664 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f27d80039c0 0x7f27d8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:59.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.047+0000 7f27eeffd640 1 --2- 172.21.15.57:0/2742745664 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f27d8004810 0x7f27d8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:59.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.047+0000 7f27eeffd640 1 --2- 172.21.15.57:0/2742745664 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f27d80052f0 0x7f27d800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:59.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.047+0000 7f27eeffd640 1 -- 172.21.15.57:0/2742745664 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f27f0073540 con 0x7f27d80052f0 2024-04-25T23:48:59.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.047+0000 7f27eeffd640 1 -- 172.21.15.57:0/2742745664 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f27d80105d0 con 0x7f27d80039c0 2024-04-25T23:48:59.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.047+0000 7f27eeffd640 1 -- 172.21.15.57:0/2742745664 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f27d80107d0 con 0x7f27d8004810 2024-04-25T23:48:59.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.047+0000 7f27ef7fe640 1 --2- 172.21.15.57:0/2742745664 >> v2:172.21.15.80:3300/0 conn(0x7f27f00b1670 0x7f27f0140340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T23:48:59.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.047+0000 7f27f4e4e640 1 --2- 172.21.15.57:0/2742745664 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f27d80052f0 0x7f27d800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:59.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.047+0000 7f27ef7fe640 1 --2- 172.21.15.57:0/2742745664 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f27d8004810 0x7f27d8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:59.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.047+0000 7f27effff640 1 --2- 172.21.15.57:0/2742745664 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f27d80039c0 0x7f27d8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:59.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.047+0000 7f27f4e4e640 1 -- 172.21.15.57:0/2742745664 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f27d8004810 msgr2=0x7f27d8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:48:59.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.047+0000 7f27f4e4e640 1 --2- 172.21.15.57:0/2742745664 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f27d8004810 0x7f27d8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:59.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.047+0000 7f27f4e4e640 1 -- 172.21.15.57:0/2742745664 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f27d80039c0 msgr2=0x7f27d8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:48:59.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.047+0000 7f27f4e4e640 1 --2- 172.21.15.57:0/2742745664 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f27d80039c0 0x7f27d8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:59.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.047+0000 7f27f4e4e640 1 -- 172.21.15.57:0/2742745664 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27d8003530 con 0x7f27d80052f0 2024-04-25T23:48:59.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.051+0000 7f27f4e4e640 1 -- 172.21.15.57:0/2742745664 reap_dead start 2024-04-25T23:48:59.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.051+0000 7f27f4e4e640 1 --2- 172.21.15.57:0/2742745664 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f27d80052f0 0x7f27d800b9d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f27f006f8e0 tx=0x7f27e801d300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc8786b7d91fe7c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:48:59.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.051+0000 7f27eeffd640 1 -- 172.21.15.57:0/2742745664 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e800a480 con 0x7f27d80052f0 2024-04-25T23:48:59.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.051+0000 7f27eeffd640 1 -- 172.21.15.57:0/2742745664 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27e80a5e50 con 0x7f27d80052f0 2024-04-25T23:48:59.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.051+0000 7f27eeffd640 1 -- 172.21.15.57:0/2742745664 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e80ad670 con 0x7f27d80052f0 2024-04-25T23:48:59.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.051+0000 7f27f68b1640 1 -- 172.21.15.57:0/2742745664 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f27d80052f0 msgr2=0x7f27d800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:59.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.051+0000 7f27f68b1640 1 --2- 172.21.15.57:0/2742745664 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f27d80052f0 0x7f27d800b9d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f27f006f8e0 tx=0x7f27e801d300 comp rx=0 tx=0).stop 2024-04-25T23:48:59.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.051+0000 7f27f68b1640 1 -- 172.21.15.57:0/2742745664 shutdown_connections 2024-04-25T23:48:59.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.051+0000 7f27f68b1640 1 --2- 172.21.15.57:0/2742745664 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f27d80052f0 0x7f27d800b9d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:59.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.051+0000 7f27f68b1640 1 -- 172.21.15.57:0/2742745664 >> 172.21.15.57:0/2742745664 conn(0x7f27f00b4740 msgr2=0x7f27f00b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:48:59.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.051+0000 7f27f68b1640 1 -- 172.21.15.57:0/2742745664 shutdown_connections 2024-04-25T23:48:59.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.051+0000 7f27f68b1640 1 -- 172.21.15.57:0/2742745664 wait complete. 2024-04-25T23:48:59.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.051+0000 7f27f68b1640 1 Processor -- start 2024-04-25T23:48:59.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.051+0000 7f27f68b1640 1 -- start start 2024-04-25T23:48:59.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.055+0000 7f27f68b1640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f27d80052f0 0x7f27f0132cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:59.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.055+0000 7f27f68b1640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f27f0139990 0x7f27f01d99f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:59.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.055+0000 7f27f68b1640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f27f01d9f30 0x7f27f012f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:59.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.055+0000 7f27f68b1640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f27f01406d0 con 0x7f27f0139990 2024-04-25T23:48:59.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.055+0000 7f27f68b1640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f27f0140550 con 0x7f27d80052f0 2024-04-25T23:48:59.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.055+0000 7f27f68b1640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f27f0139e40 con 0x7f27f01d9f30 2024-04-25T23:48:59.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.055+0000 7f27f4e4e640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f27f01d9f30 0x7f27f012f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:59.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.055+0000 7f27ef7fe640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f27f0139990 0x7f27f01d99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:59.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.055+0000 7f27f4e4e640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f27f01d9f30 0x7f27f012f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3301/0 says I am v2:172.21.15.57:47200/0 (socket says 172.21.15.57:47200) 2024-04-25T23:48:59.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.055+0000 7f27f4e4e640 1 -- 172.21.15.57:0/3461997939 learned_addr learned my addr 172.21.15.57:0/3461997939 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:48:59.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.055+0000 7f27ef7fe640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f27f0139990 0x7f27f01d99f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33302/0 (socket says 172.21.15.57:33302) 2024-04-25T23:48:59.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.055+0000 7f27effff640 1 --2- 172.21.15.57:0/3461997939 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f27d80052f0 0x7f27f0132cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:59.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.055+0000 7f27f4e4e640 1 -- 172.21.15.57:0/3461997939 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f27d80052f0 msgr2=0x7f27f0132cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:59.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.055+0000 7f27f4e4e640 1 --2- 172.21.15.57:0/3461997939 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f27d80052f0 0x7f27f0132cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:59.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.055+0000 7f27f4e4e640 1 -- 172.21.15.57:0/3461997939 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f27f0139990 msgr2=0x7f27f01d99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:59.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.055+0000 7f27f4e4e640 1 --2- 172.21.15.57:0/3461997939 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f27f0139990 0x7f27f01d99f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:59.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.055+0000 7f27f4e4e640 1 -- 172.21.15.57:0/3461997939 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27f0137900 con 0x7f27f01d9f30 2024-04-25T23:48:59.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.055+0000 7f27ef7fe640 1 --2- 172.21.15.57:0/3461997939 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f27f0139990 0x7f27f01d99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T23:48:59.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.055+0000 7f27f4e4e640 1 --2- 172.21.15.57:0/3461997939 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f27f01d9f30 0x7f27f012f0e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f27e801d130 tx=0x7f27d8005840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:48:59.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.055+0000 7f27ecff9640 1 -- 172.21.15.57:0/3461997939 <== mon.2 v2:172.21.15.57:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e80bc070 con 0x7f27f01d9f30 2024-04-25T23:48:59.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.055+0000 7f27ecff9640 1 -- 172.21.15.57:0/3461997939 <== mon.2 v2:172.21.15.57:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27e80019c0 con 0x7f27f01d9f30 2024-04-25T23:48:59.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.055+0000 7f27f68b1640 1 -- 172.21.15.57:0/3461997939 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27f0137be0 con 0x7f27f01d9f30 2024-04-25T23:48:59.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.055+0000 7f27ecff9640 1 -- 172.21.15.57:0/3461997939 <== mon.2 v2:172.21.15.57:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e80ad650 con 0x7f27f01d9f30 2024-04-25T23:48:59.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.055+0000 7f27f68b1640 1 -- 172.21.15.57:0/3461997939 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f27f0136b40 con 0x7f27f01d9f30 2024-04-25T23:48:59.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.055+0000 7f27f68b1640 1 -- 172.21.15.57:0/3461997939 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27f01435e0 con 0x7f27f01d9f30 2024-04-25T23:48:59.047 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.055+0000 7f27ecff9640 1 -- 172.21.15.57:0/3461997939 <== mon.2 v2:172.21.15.57:3301/0 4 ==== mgrmap(e 4) ==== 87354+0+0 (secure 0 0 0) 0x7f27e80bf070 con 0x7f27f01d9f30 2024-04-25T23:48:59.047 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.055+0000 7f27ecff9640 1 --2- 172.21.15.57:0/3461997939 >> v2:172.21.15.80:6816/2493457677 conn(0x7f27dc0677f0 0x7f27dc069cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:59.047 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.055+0000 7f27ecff9640 1 -- 172.21.15.57:0/3461997939 <== mon.2 v2:172.21.15.57:3301/0 5 ==== osd_map(16..16 src has 1..16) ==== 4090+0+0 (secure 0 0 0) 0x7f27e811f020 con 0x7f27f01d9f30 2024-04-25T23:48:59.047 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.055+0000 7f27effff640 1 --2- 172.21.15.57:0/3461997939 >> v2:172.21.15.80:6816/2493457677 conn(0x7f27dc0677f0 0x7f27dc069cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:59.047 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.059+0000 7f27effff640 1 --2- 172.21.15.57:0/3461997939 >> v2:172.21.15.80:6816/2493457677 conn(0x7f27dc0677f0 0x7f27dc069cb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f27e0005410 tx=0x7f27e0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:48:59.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.059+0000 7f27ecff9640 1 -- 172.21.15.57:0/3461997939 <== mon.2 v2:172.21.15.57:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f27e80b4070 con 0x7f27f01d9f30 2024-04-25T23:48:59.264 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.275+0000 7f27f68b1640 1 -- 172.21.15.57:0/3461997939 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f27f0068a10 con 0x7f27f01d9f30 2024-04-25T23:48:59.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.279+0000 7f27ecff9640 1 -- 172.21.15.57:0/3461997939 <== mon.2 v2:172.21.15.57:3301/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v4) ==== 74+0+166733 (secure 0 0 0) 0x7f27e8017010 con 0x7f27f01d9f30 2024-04-25T23:48:59.269 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-25T23:48:59.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.283+0000 7f27f68b1640 1 -- 172.21.15.57:0/3461997939 >> v2:172.21.15.80:6816/2493457677 conn(0x7f27dc0677f0 msgr2=0x7f27dc069cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:59.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.283+0000 7f27f68b1640 1 --2- 172.21.15.57:0/3461997939 >> v2:172.21.15.80:6816/2493457677 conn(0x7f27dc0677f0 0x7f27dc069cb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f27e0005410 tx=0x7f27e0009040 comp rx=0 tx=0).stop 2024-04-25T23:48:59.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.283+0000 7f27f68b1640 1 -- 172.21.15.57:0/3461997939 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f27f01d9f30 msgr2=0x7f27f012f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:59.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.283+0000 7f27f68b1640 1 --2- 172.21.15.57:0/3461997939 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f27f01d9f30 0x7f27f012f0e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f27e801d130 tx=0x7f27d8005840 comp rx=0 tx=0).stop 2024-04-25T23:48:59.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.283+0000 7f27f68b1640 1 -- 172.21.15.57:0/3461997939 shutdown_connections 2024-04-25T23:48:59.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.283+0000 7f27f68b1640 1 --2- 172.21.15.57:0/3461997939 >> v2:172.21.15.80:6816/2493457677 conn(0x7f27dc0677f0 0x7f27dc069cb0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:59.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.283+0000 7f27f68b1640 1 --2- 172.21.15.57:0/3461997939 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f27f01d9f30 0x7f27f012f0e0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:59.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.283+0000 7f27f68b1640 1 --2- 172.21.15.57:0/3461997939 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f27f0139990 0x7f27f01d99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:59.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.283+0000 7f27f68b1640 1 --2- 172.21.15.57:0/3461997939 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f27d80052f0 0x7f27f0132cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:59.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.283+0000 7f27f68b1640 1 -- 172.21.15.57:0/3461997939 >> 172.21.15.57:0/3461997939 conn(0x7f27f00b4740 msgr2=0x7f27f00ae980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:48:59.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.283+0000 7f27f68b1640 1 -- 172.21.15.57:0/3461997939 shutdown_connections 2024-04-25T23:48:59.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.283+0000 7f27f68b1640 1 -- 172.21.15.57:0/3461997939 wait complete. 2024-04-25T23:48:59.288 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":4,"flags":0,"active_gid":4100,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2493457677}]},"active_addr":"172.21.15.80:6816/2493457677","active_change":"2024-04-25T23:48:50.808581+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4104,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.80:0","nonce":950417772}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.80:0","nonce":4153406379}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.80:0","nonce":1039525001}]}]} 2024-04-25T23:48:59.297 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-25T23:48:59.297 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-25T23:48:59.297 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T23:48:59.415 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.427+0000 7f4f9b0d7640 1 Processor -- start 2024-04-25T23:48:59.415 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.427+0000 7f4f9b0d7640 1 -- start start 2024-04-25T23:48:59.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.427+0000 7f4f9b0d7640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4f940b1f00 0x7f4f941426f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:59.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.427+0000 7f4f9b0d7640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f4f941432e0 0x7f4f9406c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:59.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.427+0000 7f4f9b0d7640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f9406b920 con 0x7f4f940b1b20 2024-04-25T23:48:59.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.427+0000 7f4f9b0d7640 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 -- 0x7f4f94142e50 con 0x7f4f941432e0 2024-04-25T23:48:59.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.427+0000 7f4f9b0d7640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f4f9406d0c0 con 0x7f4f940b1f00 2024-04-25T23:48:59.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.427+0000 7f4f99674640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f4f941432e0 0x7f4f9406c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:59.417 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.427+0000 7f4f93fff640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4f940b1f00 0x7f4f941426f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:59.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.427+0000 7f4f98e73640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f4f940b1b20 0x7f4f9406b210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:49594/0 (socket says 172.21.15.57:49594) 2024-04-25T23:48:59.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.427+0000 7f4f99674640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f4f941432e0 0x7f4f9406c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33308/0 (socket says 172.21.15.57:33308) 2024-04-25T23:48:59.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.427+0000 7f4f93fff640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4f940b1f00 0x7f4f941426f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3301/0 says I am v2:172.21.15.57:47206/0 (socket says 172.21.15.57:47206) 2024-04-25T23:48:59.418 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.427+0000 7f4f98e73640 1 -- 172.21.15.57:0/1340282784 learned_addr learned my addr 172.21.15.57:0/1340282784 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-04-25T23:48:59.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.427+0000 7f4f99674640 1 -- 172.21.15.57:0/1340282784 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4f940b1f00 msgr2=0x7f4f941426f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:48:59.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.427+0000 7f4f99674640 1 --2- 172.21.15.57:0/1340282784 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4f940b1f00 0x7f4f941426f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:59.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.427+0000 7f4f99674640 1 -- 172.21.15.57:0/1340282784 >> v1:172.21.15.57:6789/0 conn(0x7f4f940b1b20 legacy=0x7f4f9406b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:59.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.427+0000 7f4f99674640 1 -- 172.21.15.57:0/1340282784 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f940b3730 con 0x7f4f941432e0 2024-04-25T23:48:59.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.427+0000 7f4f937fe640 1 -- 172.21.15.57:0/1340282784 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2641086854 0 0) 0x7f4f9406b920 con 0x7f4f940b1b20 2024-04-25T23:48:59.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.427+0000 7f4f93fff640 1 --2- 172.21.15.57:0/1340282784 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4f940b1f00 0x7f4f941426f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T23:48:59.420 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.431+0000 7f4f99674640 1 --2- 172.21.15.57:0/1340282784 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f941432e0 0x7f4f9406c060 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c009d90 tx=0x7f4f8c01c770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42e4239f942e35d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:48:59.420 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.431+0000 7f4f937fe640 1 -- 172.21.15.57:0/1340282784 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c0b2070 con 0x7f4f941432e0 2024-04-25T23:48:59.420 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.431+0000 7f4f937fe640 1 -- 172.21.15.57:0/1340282784 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f8c0a5740 con 0x7f4f941432e0 2024-04-25T23:48:59.420 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.431+0000 7f4f937fe640 1 -- 172.21.15.57:0/1340282784 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c0a5b10 con 0x7f4f941432e0 2024-04-25T23:48:59.420 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.431+0000 7f4f9b0d7640 1 -- 172.21.15.57:0/1340282784 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f941432e0 msgr2=0x7f4f9406c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:59.420 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.431+0000 7f4f9b0d7640 1 --2- 172.21.15.57:0/1340282784 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f941432e0 0x7f4f9406c060 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c009d90 tx=0x7f4f8c01c770 comp rx=0 tx=0).stop 2024-04-25T23:48:59.420 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.431+0000 7f4f9b0d7640 1 -- 172.21.15.57:0/1340282784 shutdown_connections 2024-04-25T23:48:59.420 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.431+0000 7f4f9b0d7640 1 --2- 172.21.15.57:0/1340282784 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4f940b1f00 0x7f4f941426f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:59.420 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.431+0000 7f4f9b0d7640 1 --2- 172.21.15.57:0/1340282784 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f941432e0 0x7f4f9406c060 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:59.421 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.431+0000 7f4f9b0d7640 1 -- 172.21.15.57:0/1340282784 >> 172.21.15.57:0/1340282784 conn(0x7f4f940b2c10 msgr2=0x7f4f940b3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:48:59.421 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.431+0000 7f4f9b0d7640 1 -- 172.21.15.57:0/1340282784 shutdown_connections 2024-04-25T23:48:59.421 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.431+0000 7f4f9b0d7640 1 -- 172.21.15.57:0/1340282784 wait complete. 2024-04-25T23:48:59.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.431+0000 7f4f9b0d7640 1 Processor -- start 2024-04-25T23:48:59.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.431+0000 7f4f9b0d7640 1 -- start start 2024-04-25T23:48:59.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.435+0000 7f4f9b0d7640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4f940b1b20 0x7f4f9413aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:59.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.435+0000 7f4f9b0d7640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f941432e0 0x7f4f940aede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:59.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.435+0000 7f4f9b0d7640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4f940afa60 0x7f4f940afef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:59.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.435+0000 7f4f9b0d7640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9414fad0 con 0x7f4f941432e0 2024-04-25T23:48:59.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.435+0000 7f4f9b0d7640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9414f950 con 0x7f4f940b1b20 2024-04-25T23:48:59.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.435+0000 7f4f9b0d7640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f4f9414fc50 con 0x7f4f940afa60 2024-04-25T23:48:59.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.435+0000 7f4f93fff640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f941432e0 0x7f4f940aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:59.424 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.435+0000 7f4f93fff640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f941432e0 0x7f4f940aede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33310/0 (socket says 172.21.15.57:33310) 2024-04-25T23:48:59.424 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.435+0000 7f4f93fff640 1 -- 172.21.15.57:0/650997203 learned_addr learned my addr 172.21.15.57:0/650997203 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:48:59.424 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.435+0000 7f4f99674640 1 --2- 172.21.15.57:0/650997203 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4f940afa60 0x7f4f940afef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:59.424 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.435+0000 7f4f98e73640 1 --2- 172.21.15.57:0/650997203 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4f940b1b20 0x7f4f9413aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:59.424 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.435+0000 7f4f93fff640 1 -- 172.21.15.57:0/650997203 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4f940afa60 msgr2=0x7f4f940afef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:59.424 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.435+0000 7f4f93fff640 1 --2- 172.21.15.57:0/650997203 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4f940afa60 0x7f4f940afef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:59.424 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.435+0000 7f4f93fff640 1 -- 172.21.15.57:0/650997203 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4f940b1b20 msgr2=0x7f4f9413aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:59.424 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.435+0000 7f4f93fff640 1 --2- 172.21.15.57:0/650997203 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4f940b1b20 0x7f4f9413aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:59.424 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.435+0000 7f4f93fff640 1 -- 172.21.15.57:0/650997203 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f940af540 con 0x7f4f941432e0 2024-04-25T23:48:59.424 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.435+0000 7f4f99674640 1 --2- 172.21.15.57:0/650997203 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4f940afa60 0x7f4f940afef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T23:48:59.424 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.435+0000 7f4f93fff640 1 --2- 172.21.15.57:0/650997203 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f941432e0 0x7f4f940aede0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4f8000e980 tx=0x7f4f8000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:48:59.424 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.435+0000 7f4f917fa640 1 -- 172.21.15.57:0/650997203 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f80017070 con 0x7f4f941432e0 2024-04-25T23:48:59.424 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.435+0000 7f4f917fa640 1 -- 172.21.15.57:0/650997203 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f8000ca00 con 0x7f4f941432e0 2024-04-25T23:48:59.424 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.435+0000 7f4f9b0d7640 1 -- 172.21.15.57:0/650997203 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f94144050 con 0x7f4f941432e0 2024-04-25T23:48:59.424 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.435+0000 7f4f917fa640 1 -- 172.21.15.57:0/650997203 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8000cdd0 con 0x7f4f941432e0 2024-04-25T23:48:59.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.435+0000 7f4f9b0d7640 1 -- 172.21.15.57:0/650997203 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f941ddff0 con 0x7f4f941432e0 2024-04-25T23:48:59.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.435+0000 7f4f9b0d7640 1 -- 172.21.15.57:0/650997203 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f94068210 con 0x7f4f941432e0 2024-04-25T23:48:59.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.435+0000 7f4f917fa640 1 -- 172.21.15.57:0/650997203 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 4) ==== 87354+0+0 (secure 0 0 0) 0x7f4f8001a070 con 0x7f4f941432e0 2024-04-25T23:48:59.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.435+0000 7f4f917fa640 1 --2- 172.21.15.57:0/650997203 >> v2:172.21.15.80:6816/2493457677 conn(0x7f4f78067840 0x7f4f78069d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:59.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.435+0000 7f4f917fa640 1 -- 172.21.15.57:0/650997203 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4090+0+0 (secure 0 0 0) 0x7f4f80085240 con 0x7f4f941432e0 2024-04-25T23:48:59.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.435+0000 7f4f98e73640 1 --2- 172.21.15.57:0/650997203 >> v2:172.21.15.80:6816/2493457677 conn(0x7f4f78067840 0x7f4f78069d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:59.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.439+0000 7f4f98e73640 1 --2- 172.21.15.57:0/650997203 >> v2:172.21.15.80:6816/2493457677 conn(0x7f4f78067840 0x7f4f78069d00 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4f88000f70 tx=0x7f4f88007c10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:48:59.428 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.439+0000 7f4f917fa640 1 -- 172.21.15.57:0/650997203 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4f80055fd0 con 0x7f4f941432e0 2024-04-25T23:48:59.592 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.603+0000 7f4f9b0d7640 1 -- 172.21.15.57:0/650997203 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4f9406c380 con 0x7f4f941432e0 2024-04-25T23:48:59.592 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.603+0000 7f4f917fa640 1 -- 172.21.15.57:0/650997203 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12765 (secure 0 0 0) 0x7f4f8005a2d0 con 0x7f4f941432e0 2024-04-25T23:48:59.593 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-25T23:48:59.593 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":16,"fsid":"ca0adda3-98dd-4ba5-b4b2-8b4f20dcabdd","created":"2024-04-25T23:48:46.211894+0000","modified":"2024-04-25T23:48:58.879448+0000","last_up_change":"2024-04-25T23:48:56.859933+0000","last_in_change":"2024-04-25T23:48:48.838840+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T23:48:56.919796+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-25T23:48:57.576441+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5986c317-8828-4568-af68-a8ae96690c51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":2528545516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6813","nonce":2528545516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6815","nonce":2528545516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":2528545516}]},"public_addr":"172.21.15.57:6812/2528545516","cluster_addr":"172.21.15.57:6813/2528545516","heartbeat_back_addr":"172.21.15.57:6815/2528545516","heartbeat_front_addr":"172.21.15.57:6814/2528545516","state":["exists","up"]},{"osd":1,"uuid":"1aa157dd-52fe-4bb3-9f63-4db493de5333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":2225240106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6809","nonce":2225240106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6811","nonce":2225240106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":2225240106}]},"public_addr":"172.21.15.57:6808/2225240106","cluster_addr":"172.21.15.57:6809/2225240106","heartbeat_back_addr":"172.21.15.57:6811/2225240106","heartbeat_front_addr":"172.21.15.57:6810/2225240106","state":["exists","up"]},{"osd":2,"uuid":"1391be38-25ac-43f0-8090-15adddc6d69d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":1976583347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6805","nonce":1976583347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6807","nonce":1976583347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":1976583347}]},"public_addr":"172.21.15.57:6804/1976583347","cluster_addr":"172.21.15.57:6805/1976583347","heartbeat_back_addr":"172.21.15.57:6807/1976583347","heartbeat_front_addr":"172.21.15.57:6806/1976583347","state":["exists","up"]},{"osd":3,"uuid":"a85e7650-effb-4ea7-9f3b-9fc9fc1576e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":1899803014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6801","nonce":1899803014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6803","nonce":1899803014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":1899803014}]},"public_addr":"172.21.15.57:6800/1899803014","cluster_addr":"172.21.15.57:6801/1899803014","heartbeat_back_addr":"172.21.15.57:6803/1899803014","heartbeat_front_addr":"172.21.15.57:6802/1899803014","state":["exists","up"]},{"osd":4,"uuid":"ad3fb1a9-cec6-4fb3-9f49-22b9480f524b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":4069821322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":4069821322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":4069821322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":4069821322}]},"public_addr":"172.21.15.80:6800/4069821322","cluster_addr":"172.21.15.80:6802/4069821322","heartbeat_back_addr":"172.21.15.80:6806/4069821322","heartbeat_front_addr":"172.21.15.80:6804/4069821322","state":["exists","up"]},{"osd":5,"uuid":"fd05cda6-27a5-477a-8916-b34b69ba9ef6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6811","nonce":2954555099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":2954555099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6815","nonce":2954555099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2954555099}]},"public_addr":"172.21.15.80:6811/2954555099","cluster_addr":"172.21.15.80:6813/2954555099","heartbeat_back_addr":"172.21.15.80:6815/2954555099","heartbeat_front_addr":"172.21.15.80:6814/2954555099","state":["exists","up"]},{"osd":6,"uuid":"599ef421-483d-4225-a98f-7405006b1809","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6801","nonce":163896711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6803","nonce":163896711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6807","nonce":163896711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6805","nonce":163896711}]},"public_addr":"172.21.15.80:6801/163896711","cluster_addr":"172.21.15.80:6803/163896711","heartbeat_back_addr":"172.21.15.80:6807/163896711","heartbeat_front_addr":"172.21.15.80:6805/163896711","state":["exists","up"]},{"osd":7,"uuid":"0e9d533c-465f-403c-8fe8-7dafc953d774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2647667231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":2647667231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2647667231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2647667231}]},"public_addr":"172.21.15.80:6808/2647667231","cluster_addr":"172.21.15.80:6809/2647667231","heartbeat_back_addr":"172.21.15.80:6812/2647667231","heartbeat_front_addr":"172.21.15.80:6810/2647667231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T23:48:54.460484+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T23:48:54.420671+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T23:48:54.335197+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T23:48:54.243121+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T23:48:54.249670+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T23:48:54.153679+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T23:48:53.992685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T23:48:54.129310+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T23:48:59.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.607+0000 7f4f9b0d7640 1 -- 172.21.15.57:0/650997203 >> v2:172.21.15.80:6816/2493457677 conn(0x7f4f78067840 msgr2=0x7f4f78069d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:59.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.607+0000 7f4f9b0d7640 1 --2- 172.21.15.57:0/650997203 >> v2:172.21.15.80:6816/2493457677 conn(0x7f4f78067840 0x7f4f78069d00 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4f88000f70 tx=0x7f4f88007c10 comp rx=0 tx=0).stop 2024-04-25T23:48:59.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.607+0000 7f4f9b0d7640 1 -- 172.21.15.57:0/650997203 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f941432e0 msgr2=0x7f4f940aede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:59.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.607+0000 7f4f9b0d7640 1 --2- 172.21.15.57:0/650997203 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f941432e0 0x7f4f940aede0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4f8000e980 tx=0x7f4f8000ee40 comp rx=0 tx=0).stop 2024-04-25T23:48:59.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.607+0000 7f4f9b0d7640 1 -- 172.21.15.57:0/650997203 shutdown_connections 2024-04-25T23:48:59.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.607+0000 7f4f9b0d7640 1 --2- 172.21.15.57:0/650997203 >> v2:172.21.15.80:6816/2493457677 conn(0x7f4f78067840 0x7f4f78069d00 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:59.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.607+0000 7f4f9b0d7640 1 --2- 172.21.15.57:0/650997203 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4f940afa60 0x7f4f940afef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:59.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.607+0000 7f4f9b0d7640 1 --2- 172.21.15.57:0/650997203 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f941432e0 0x7f4f940aede0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:59.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.607+0000 7f4f9b0d7640 1 --2- 172.21.15.57:0/650997203 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4f940b1b20 0x7f4f9413aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:59.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.607+0000 7f4f9b0d7640 1 -- 172.21.15.57:0/650997203 >> 172.21.15.57:0/650997203 conn(0x7f4f940b2c10 msgr2=0x7f4f94071ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:48:59.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.607+0000 7f4f9b0d7640 1 -- 172.21.15.57:0/650997203 shutdown_connections 2024-04-25T23:48:59.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.607+0000 7f4f9b0d7640 1 -- 172.21.15.57:0/650997203 wait complete. 2024-04-25T23:48:59.612 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-25T23:48:59.612 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T23:48:59.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.739+0000 7ff4c198d640 1 Processor -- start 2024-04-25T23:48:59.728 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.739+0000 7ff4c198d640 1 -- start start 2024-04-25T23:48:59.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.739+0000 7ff4c198d640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7ff4bc06bbe0 0x7ff4bc06bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:59.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.739+0000 7ff4c198d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff4bc06cd50 0x7ff4bc146100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:59.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.739+0000 7ff4c198d640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4bc0b2940 con 0x7ff4bc071b30 2024-04-25T23:48:59.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.739+0000 7ff4c198d640 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 -- 0x7ff4bc0714c0 con 0x7ff4bc06bbe0 2024-04-25T23:48:59.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.739+0000 7ff4c198d640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7ff4bc145be0 con 0x7ff4bc06cd50 2024-04-25T23:48:59.729 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.739+0000 7ff4baffd640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7ff4bc06bbe0 0x7ff4bc06bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:59.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.739+0000 7ff4baffd640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7ff4bc06bbe0 0x7ff4bc06bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33316/0 (socket says 172.21.15.57:33316) 2024-04-25T23:48:59.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.739+0000 7ff4baffd640 1 -- 172.21.15.57:0/125467946 learned_addr learned my addr 172.21.15.57:0/125467946 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:48:59.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.739+0000 7ff4bb7fe640 1 --2- 172.21.15.57:0/125467946 >> v2:172.21.15.80:3300/0 conn(0x7ff4bc06cd50 0x7ff4bc146100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:59.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.743+0000 7ff4baffd640 1 -- 172.21.15.57:0/125467946 >> v2:172.21.15.80:3300/0 conn(0x7ff4bc06cd50 msgr2=0x7ff4bc146100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:48:59.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.743+0000 7ff4baffd640 1 --2- 172.21.15.57:0/125467946 >> v2:172.21.15.80:3300/0 conn(0x7ff4bc06cd50 0x7ff4bc146100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:59.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.743+0000 7ff4baffd640 1 -- 172.21.15.57:0/125467946 >> v1:172.21.15.80:6789/0 conn(0x7ff4bc071b30 legacy=0x7ff4bc13fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:59.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.743+0000 7ff4baffd640 1 -- 172.21.15.57:0/125467946 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4bc140110 con 0x7ff4bc06bbe0 2024-04-25T23:48:59.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.743+0000 7ff4baffd640 1 --2- 172.21.15.57:0/125467946 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff4bc06bbe0 0x7ff4bc06bfc0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc073850 tx=0x7ff4ac01c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe715167896d9732 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:48:59.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.743+0000 7ff4b9ffb640 1 -- 172.21.15.57:0/125467946 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac0aa680 con 0x7ff4bc06bbe0 2024-04-25T23:48:59.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.743+0000 7ff4b9ffb640 1 -- 172.21.15.57:0/125467946 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4ac0af070 con 0x7ff4bc06bbe0 2024-04-25T23:48:59.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.743+0000 7ff4b9ffb640 1 -- 172.21.15.57:0/125467946 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac0aaa40 con 0x7ff4bc06bbe0 2024-04-25T23:48:59.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.743+0000 7ff4c198d640 1 -- 172.21.15.57:0/125467946 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff4bc06bbe0 msgr2=0x7ff4bc06bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:59.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.743+0000 7ff4c198d640 1 --2- 172.21.15.57:0/125467946 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff4bc06bbe0 0x7ff4bc06bfc0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc073850 tx=0x7ff4ac01c500 comp rx=0 tx=0).stop 2024-04-25T23:48:59.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.743+0000 7ff4c198d640 1 -- 172.21.15.57:0/125467946 shutdown_connections 2024-04-25T23:48:59.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.743+0000 7ff4c198d640 1 --2- 172.21.15.57:0/125467946 >> v2:172.21.15.80:3300/0 conn(0x7ff4bc06cd50 0x7ff4bc146100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:59.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.743+0000 7ff4c198d640 1 --2- 172.21.15.57:0/125467946 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff4bc06bbe0 0x7ff4bc06bfc0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:59.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.743+0000 7ff4c198d640 1 -- 172.21.15.57:0/125467946 >> 172.21.15.57:0/125467946 conn(0x7ff4bc0b60d0 msgr2=0x7ff4bc0b64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:48:59.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.743+0000 7ff4c198d640 1 -- 172.21.15.57:0/125467946 shutdown_connections 2024-04-25T23:48:59.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.743+0000 7ff4c198d640 1 -- 172.21.15.57:0/125467946 wait complete. 2024-04-25T23:48:59.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.747+0000 7ff4c198d640 1 Processor -- start 2024-04-25T23:48:59.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.747+0000 7ff4c198d640 1 -- start start 2024-04-25T23:48:59.735 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.747+0000 7ff4c198d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4bc06bbe0 0x7ff4bc140ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:59.736 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.747+0000 7ff4c198d640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff4bc06cd50 0x7ff4bc141730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:59.736 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.747+0000 7ff4c198d640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7ff4bc071b30 0x7ff4bc132470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:59.736 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.747+0000 7ff4c198d640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7ff4bc14fa60 con 0x7ff4bc06cd50 2024-04-25T23:48:59.736 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.747+0000 7ff4c198d640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff4bc14f8e0 con 0x7ff4bc06bbe0 2024-04-25T23:48:59.736 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.747+0000 7ff4c198d640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7ff4bc14fbe0 con 0x7ff4bc071b30 2024-04-25T23:48:59.736 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.747+0000 7ff4bb7fe640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7ff4bc071b30 0x7ff4bc132470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:59.736 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.747+0000 7ff4ba7fc640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff4bc06cd50 0x7ff4bc141730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:59.736 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.747+0000 7ff4bb7fe640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7ff4bc071b30 0x7ff4bc132470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3301/0 says I am v2:172.21.15.57:47230/0 (socket says 172.21.15.57:47230) 2024-04-25T23:48:59.736 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.747+0000 7ff4bb7fe640 1 -- 172.21.15.57:0/395146706 learned_addr learned my addr 172.21.15.57:0/395146706 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:48:59.736 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.747+0000 7ff4baffd640 1 --2- 172.21.15.57:0/395146706 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4bc06bbe0 0x7ff4bc140ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:59.736 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.747+0000 7ff4bb7fe640 1 -- 172.21.15.57:0/395146706 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4bc06bbe0 msgr2=0x7ff4bc140ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:59.736 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.747+0000 7ff4bb7fe640 1 --2- 172.21.15.57:0/395146706 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4bc06bbe0 0x7ff4bc140ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:59.736 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.747+0000 7ff4bb7fe640 1 -- 172.21.15.57:0/395146706 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff4bc06cd50 msgr2=0x7ff4bc141730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:59.736 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.747+0000 7ff4bb7fe640 1 --2- 172.21.15.57:0/395146706 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff4bc06cd50 0x7ff4bc141730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:59.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.747+0000 7ff4bb7fe640 1 -- 172.21.15.57:0/395146706 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4bc143180 con 0x7ff4bc071b30 2024-04-25T23:48:59.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.747+0000 7ff4ba7fc640 1 --2- 172.21.15.57:0/395146706 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff4bc06cd50 0x7ff4bc141730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T23:48:59.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.747+0000 7ff4bb7fe640 1 --2- 172.21.15.57:0/395146706 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7ff4bc071b30 0x7ff4bc132470 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff4b400efc0 tx=0x7ff4b400c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:48:59.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.747+0000 7ff4a3fff640 1 -- 172.21.15.57:0/395146706 <== mon.2 v2:172.21.15.57:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b4018020 con 0x7ff4bc071b30 2024-04-25T23:48:59.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.747+0000 7ff4a3fff640 1 -- 172.21.15.57:0/395146706 <== mon.2 v2:172.21.15.57:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4b4014480 con 0x7ff4bc071b30 2024-04-25T23:48:59.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.747+0000 7ff4a3fff640 1 -- 172.21.15.57:0/395146706 <== mon.2 v2:172.21.15.57:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b4014870 con 0x7ff4bc071b30 2024-04-25T23:48:59.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.747+0000 7ff4c198d640 1 -- 172.21.15.57:0/395146706 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4bc1433f0 con 0x7ff4bc071b30 2024-04-25T23:48:59.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.747+0000 7ff4c198d640 1 -- 172.21.15.57:0/395146706 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4bc1329b0 con 0x7ff4bc071b30 2024-04-25T23:48:59.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.747+0000 7ff4c198d640 1 -- 172.21.15.57:0/395146706 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4bc068210 con 0x7ff4bc071b30 2024-04-25T23:48:59.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.751+0000 7ff4a3fff640 1 -- 172.21.15.57:0/395146706 <== mon.2 v2:172.21.15.57:3301/0 4 ==== mgrmap(e 4) ==== 87354+0+0 (secure 0 0 0) 0x7ff4b401b020 con 0x7ff4bc071b30 2024-04-25T23:48:59.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.751+0000 7ff4a3fff640 1 --2- 172.21.15.57:0/395146706 >> v2:172.21.15.80:6816/2493457677 conn(0x7ff4a8067820 0x7ff4a8069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:48:59.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.751+0000 7ff4a3fff640 1 -- 172.21.15.57:0/395146706 <== mon.2 v2:172.21.15.57:3301/0 5 ==== osd_map(16..16 src has 1..16) ==== 4090+0+0 (secure 0 0 0) 0x7ff4b4085ee0 con 0x7ff4bc071b30 2024-04-25T23:48:59.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.751+0000 7ff4baffd640 1 --2- 172.21.15.57:0/395146706 >> v2:172.21.15.80:6816/2493457677 conn(0x7ff4a8067820 0x7ff4a8069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:48:59.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.751+0000 7ff4baffd640 1 --2- 172.21.15.57:0/395146706 >> v2:172.21.15.80:6816/2493457677 conn(0x7ff4a8067820 0x7ff4a8069ce0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac0ae000 tx=0x7ff4ac0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:48:59.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.755+0000 7ff4a3fff640 1 -- 172.21.15.57:0/395146706 <== mon.2 v2:172.21.15.57:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff4b4022070 con 0x7ff4bc071b30 2024-04-25T23:48:59.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.931+0000 7ff4c198d640 1 -- 172.21.15.57:0/395146706 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff4bc06c860 con 0x7ff4bc071b30 2024-04-25T23:48:59.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.935+0000 7ff4a3fff640 1 -- 172.21.15.57:0/395146706 <== mon.2 v2:172.21.15.57:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12739 (secure 0 0 0) 0x7ff4b4010040 con 0x7ff4bc071b30 2024-04-25T23:48:59.921 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-25T23:48:59.922 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":17,"fsid":"ca0adda3-98dd-4ba5-b4b2-8b4f20dcabdd","created":"2024-04-25T23:48:46.211894+0000","modified":"2024-04-25T23:48:59.890274+0000","last_up_change":"2024-04-25T23:48:56.859933+0000","last_in_change":"2024-04-25T23:48:48.838840+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T23:48:56.919796+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-25T23:48:57.576441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5986c317-8828-4568-af68-a8ae96690c51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":2528545516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6813","nonce":2528545516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6815","nonce":2528545516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":2528545516}]},"public_addr":"172.21.15.57:6812/2528545516","cluster_addr":"172.21.15.57:6813/2528545516","heartbeat_back_addr":"172.21.15.57:6815/2528545516","heartbeat_front_addr":"172.21.15.57:6814/2528545516","state":["exists","up"]},{"osd":1,"uuid":"1aa157dd-52fe-4bb3-9f63-4db493de5333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":2225240106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6809","nonce":2225240106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6811","nonce":2225240106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":2225240106}]},"public_addr":"172.21.15.57:6808/2225240106","cluster_addr":"172.21.15.57:6809/2225240106","heartbeat_back_addr":"172.21.15.57:6811/2225240106","heartbeat_front_addr":"172.21.15.57:6810/2225240106","state":["exists","up"]},{"osd":2,"uuid":"1391be38-25ac-43f0-8090-15adddc6d69d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":1976583347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6805","nonce":1976583347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6807","nonce":1976583347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":1976583347}]},"public_addr":"172.21.15.57:6804/1976583347","cluster_addr":"172.21.15.57:6805/1976583347","heartbeat_back_addr":"172.21.15.57:6807/1976583347","heartbeat_front_addr":"172.21.15.57:6806/1976583347","state":["exists","up"]},{"osd":3,"uuid":"a85e7650-effb-4ea7-9f3b-9fc9fc1576e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":1899803014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6801","nonce":1899803014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6803","nonce":1899803014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":1899803014}]},"public_addr":"172.21.15.57:6800/1899803014","cluster_addr":"172.21.15.57:6801/1899803014","heartbeat_back_addr":"172.21.15.57:6803/1899803014","heartbeat_front_addr":"172.21.15.57:6802/1899803014","state":["exists","up"]},{"osd":4,"uuid":"ad3fb1a9-cec6-4fb3-9f49-22b9480f524b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":4069821322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":4069821322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":4069821322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":4069821322}]},"public_addr":"172.21.15.80:6800/4069821322","cluster_addr":"172.21.15.80:6802/4069821322","heartbeat_back_addr":"172.21.15.80:6806/4069821322","heartbeat_front_addr":"172.21.15.80:6804/4069821322","state":["exists","up"]},{"osd":5,"uuid":"fd05cda6-27a5-477a-8916-b34b69ba9ef6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6811","nonce":2954555099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":2954555099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6815","nonce":2954555099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2954555099}]},"public_addr":"172.21.15.80:6811/2954555099","cluster_addr":"172.21.15.80:6813/2954555099","heartbeat_back_addr":"172.21.15.80:6815/2954555099","heartbeat_front_addr":"172.21.15.80:6814/2954555099","state":["exists","up"]},{"osd":6,"uuid":"599ef421-483d-4225-a98f-7405006b1809","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6801","nonce":163896711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6803","nonce":163896711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6807","nonce":163896711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6805","nonce":163896711}]},"public_addr":"172.21.15.80:6801/163896711","cluster_addr":"172.21.15.80:6803/163896711","heartbeat_back_addr":"172.21.15.80:6807/163896711","heartbeat_front_addr":"172.21.15.80:6805/163896711","state":["exists","up"]},{"osd":7,"uuid":"0e9d533c-465f-403c-8fe8-7dafc953d774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2647667231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":2647667231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2647667231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2647667231}]},"public_addr":"172.21.15.80:6808/2647667231","cluster_addr":"172.21.15.80:6809/2647667231","heartbeat_back_addr":"172.21.15.80:6812/2647667231","heartbeat_front_addr":"172.21.15.80:6810/2647667231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T23:48:54.460484+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T23:48:54.420671+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T23:48:54.335197+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T23:48:54.243121+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T23:48:54.249670+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T23:48:54.153679+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T23:48:53.992685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T23:48:54.129310+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T23:48:59.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.935+0000 7ff4c198d640 1 -- 172.21.15.57:0/395146706 >> v2:172.21.15.80:6816/2493457677 conn(0x7ff4a8067820 msgr2=0x7ff4a8069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:59.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.935+0000 7ff4c198d640 1 --2- 172.21.15.57:0/395146706 >> v2:172.21.15.80:6816/2493457677 conn(0x7ff4a8067820 0x7ff4a8069ce0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac0ae000 tx=0x7ff4ac0a7000 comp rx=0 tx=0).stop 2024-04-25T23:48:59.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.935+0000 7ff4c198d640 1 -- 172.21.15.57:0/395146706 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7ff4bc071b30 msgr2=0x7ff4bc132470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:48:59.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.935+0000 7ff4c198d640 1 --2- 172.21.15.57:0/395146706 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7ff4bc071b30 0x7ff4bc132470 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff4b400efc0 tx=0x7ff4b400c490 comp rx=0 tx=0).stop 2024-04-25T23:48:59.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.935+0000 7ff4c198d640 1 -- 172.21.15.57:0/395146706 shutdown_connections 2024-04-25T23:48:59.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.935+0000 7ff4c198d640 1 --2- 172.21.15.57:0/395146706 >> v2:172.21.15.80:6816/2493457677 conn(0x7ff4a8067820 0x7ff4a8069ce0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:59.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.935+0000 7ff4c198d640 1 --2- 172.21.15.57:0/395146706 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7ff4bc071b30 0x7ff4bc132470 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:59.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.935+0000 7ff4c198d640 1 --2- 172.21.15.57:0/395146706 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff4bc06cd50 0x7ff4bc141730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:59.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.935+0000 7ff4c198d640 1 --2- 172.21.15.57:0/395146706 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4bc06bbe0 0x7ff4bc140ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:48:59.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.935+0000 7ff4c198d640 1 -- 172.21.15.57:0/395146706 >> 172.21.15.57:0/395146706 conn(0x7ff4bc0b60d0 msgr2=0x7ff4bc0b2b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:48:59.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.935+0000 7ff4c198d640 1 -- 172.21.15.57:0/395146706 shutdown_connections 2024-04-25T23:48:59.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:48:59.935+0000 7ff4c198d640 1 -- 172.21.15.57:0/395146706 wait complete. 2024-04-25T23:48:59.941 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-25T23:48:59.942 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-25T23:48:59.942 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-25T23:48:59.943 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-25T23:48:59.943 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-25T23:48:59.943 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-25T23:48:59.944 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-25T23:48:59.944 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-25T23:49:00.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.143+0000 7f2acd379640 1 Processor -- start 2024-04-25T23:49:00.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7fbc0a926640 1 Processor -- start 2024-04-25T23:49:00.134 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7fbc0a926640 1 -- start start 2024-04-25T23:49:00.134 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7fbc0a926640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fbbfc035040 0x7fbbfc02d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.134 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7fbc0a926640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbfc02e690 con 0x7fbbfc036e30 2024-04-25T23:49:00.134 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7fbc0a926640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbfc02cb10 con 0x7fbbfc02d590 2024-04-25T23:49:00.134 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7fbc0a926640 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 -- 0x7fbbfc037210 con 0x7fbbfc035040 2024-04-25T23:49:00.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7f2acd379640 1 -- start start 2024-04-25T23:49:00.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7f2acd379640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f2ac806ee00 0x7f2ac813b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7f2acd379640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f2ac806f1e0 0x7f2ac813bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7f2acd379640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ac8071060 con 0x7f2ac813e780 2024-04-25T23:49:00.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7f2acd379640 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 -- 0x7f2ac813c020 con 0x7f2ac806f1e0 2024-04-25T23:49:00.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7f2acd379640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f2ac8071340 con 0x7f2ac806ee00 2024-04-25T23:49:00.135 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7f2accb78640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f2ac813e780 0x7f2ac813ebf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.57:39624/0 (socket says 172.21.15.57:39624) 2024-04-25T23:49:00.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7f2accb78640 1 -- 172.21.15.57:0/644564634 learned_addr learned my addr 172.21.15.57:0/644564634 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-04-25T23:49:00.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7f2ac77fe640 1 --2- 172.21.15.57:0/644564634 >> v2:172.21.15.57:3300/0 conn(0x7f2ac806f1e0 0x7f2ac813bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7fbc09924640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fbbfc036e30 0x7fbbfc0c8f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:49610/0 (socket says 172.21.15.57:49610) 2024-04-25T23:49:00.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7fbc09924640 1 -- 172.21.15.57:0/2282819448 learned_addr learned my addr 172.21.15.57:0/2282819448 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-04-25T23:49:00.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7fbc09123640 1 --2- 172.21.15.57:0/2282819448 >> v2:172.21.15.57:3300/0 conn(0x7fbbfc035040 0x7fbbfc02d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7fbc08922640 1 -- 172.21.15.57:0/2282819448 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2701523089 0 0) 0x7fbbfc02e690 con 0x7fbbfc036e30 2024-04-25T23:49:00.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7fbc08922640 1 -- 172.21.15.57:0/2282819448 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbbf0003530 con 0x7fbbfc036e30 2024-04-25T23:49:00.136 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7fa5ed990640 1 Processor -- start 2024-04-25T23:49:00.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7fa5ed990640 1 -- start start 2024-04-25T23:49:00.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7f1ed07e8640 1 Processor -- start 2024-04-25T23:49:00.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7f1ed07e8640 1 -- start start 2024-04-25T23:49:00.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7fd6e759e640 1 Processor -- start 2024-04-25T23:49:00.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7fd6e759e640 1 -- start start 2024-04-25T23:49:00.137 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7fa156c9f640 1 Processor -- start 2024-04-25T23:49:00.138 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7fbc08922640 1 -- 172.21.15.57:0/2282819448 <== mon.1 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1344107540 0 0) 0x7fbbfc02cb10 con 0x7fbbfc02d590 2024-04-25T23:49:00.138 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7fbc08922640 1 -- 172.21.15.57:0/2282819448 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbbfc02e690 con 0x7fbbfc02d590 2024-04-25T23:49:00.138 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7f2ac6ffd640 1 -- 172.21.15.57:0/644564634 <== mon.1 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1970711964 0 0) 0x7f2ac8071060 con 0x7f2ac813e780 2024-04-25T23:49:00.138 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7f2ac6ffd640 1 -- 172.21.15.57:0/644564634 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ab0003530 con 0x7f2ac813e780 2024-04-25T23:49:00.138 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7f2ac77fe640 1 -- 172.21.15.57:0/644564634 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f2ac806ee00 msgr2=0x7f2ac813b5a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T23:49:00.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7fd6e759e640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fd6e8074b50 0x7fd6e80af7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7fd6e759e640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fd6e813f3e0 0x7fd6e813f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7fd6e759e640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6e8074f30 con 0x7fd6e8142490 2024-04-25T23:49:00.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7fd6e759e640 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 -- 0x7fd6e80afe80 con 0x7fd6e8074b50 2024-04-25T23:49:00.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7fd6e759e640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7fd6e80b15a0 con 0x7fd6e813f3e0 2024-04-25T23:49:00.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7f1ed07e8640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f1ec80b51e0 0x7f1ec813f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7f1ed07e8640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f1ec80a9e50 0x7f1ec813f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7f1ed07e8640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1ec8145c60 0x7f1ec81460d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7f1ed07e8640 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 -- 0x7f1ec80b55c0 con 0x7f1ec80b51e0 2024-04-25T23:49:00.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7f1ed07e8640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f1ec813fe70 con 0x7f1ec8145c60 2024-04-25T23:49:00.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7f1ed07e8640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f1ec80ab2f0 con 0x7f1ec80a9e50 2024-04-25T23:49:00.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7f2ac7fff640 1 --2- 172.21.15.57:0/644564634 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f2ac806ee00 0x7f2ac813b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7f2ac6ffd640 1 -- 172.21.15.57:0/644564634 <== mon.1 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3160506702 0 0) 0x7f2ab0003530 con 0x7f2ac813e780 2024-04-25T23:49:00.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7fbc09123640 1 -- 172.21.15.57:0/2282819448 >> v1:172.21.15.80:6789/0 conn(0x7fbbfc02d590 legacy=0x7fbbfc0c2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7fbc09123640 1 -- 172.21.15.57:0/2282819448 >> v1:172.21.15.57:6789/0 conn(0x7fbbfc036e30 legacy=0x7fbbfc0c8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7fbc09123640 1 -- 172.21.15.57:0/2282819448 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbfc0c9610 con 0x7fbbfc035040 2024-04-25T23:49:00.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7fa5ed990640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa5e0037610 0x7fa5e00c7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7fa5ed990640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fa5e002c5e0 0x7fa5e00c1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.151+0000 7fa5ed990640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5e002d620 con 0x7fa5e002b8f0 2024-04-25T23:49:00.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.151+0000 7fa5ed990640 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 -- 0x7fa5e002b460 con 0x7fa5e002c5e0 2024-04-25T23:49:00.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.151+0000 7fa5ed990640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fa5e002d900 con 0x7fa5e0037610 2024-04-25T23:49:00.141 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7f1eced85640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1ec8145c60 0x7f1ec81460d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7f1eced85640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1ec8145c60 0x7f1ec81460d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.57:54164/0 (socket says 172.21.15.57:54164) 2024-04-25T23:49:00.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7f1eced85640 1 -- 172.21.15.57:0/3933796287 learned_addr learned my addr 172.21.15.57:0/3933796287 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:49:00.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7f1eced85640 1 -- 172.21.15.57:0/3933796287 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f1ec80a9e50 msgr2=0x7f1ec813f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:49:00.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7f1eced85640 1 --2- 172.21.15.57:0/3933796287 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f1ec80a9e50 0x7f1ec813f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7f1eced85640 1 -- 172.21.15.57:0/3933796287 >> v2:172.21.15.57:3300/0 conn(0x7f1ec80b51e0 msgr2=0x7f1ec813f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:49:00.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7f1eced85640 1 --2- 172.21.15.57:0/3933796287 >> v2:172.21.15.57:3300/0 conn(0x7f1ec80b51e0 0x7f1ec813f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7f1eced85640 1 -- 172.21.15.57:0/3933796287 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ec80ab5d0 con 0x7f1ec8145c60 2024-04-25T23:49:00.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7fd6e6d9d640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fd6e813f3e0 0x7fd6e813f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7fd6e5d9b640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fd6e8142490 0x7fd6e81428d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.57:39628/0 (socket says 172.21.15.57:39628) 2024-04-25T23:49:00.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7fd6e5d9b640 1 -- 172.21.15.57:0/110218354 learned_addr learned my addr 172.21.15.57:0/110218354 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-04-25T23:49:00.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7fd6e6d9d640 1 -- 172.21.15.57:0/110218354 >> v2:172.21.15.57:3300/0 conn(0x7fd6e8074b50 msgr2=0x7fd6e80af7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:49:00.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7fd6e6d9d640 1 --2- 172.21.15.57:0/110218354 >> v2:172.21.15.57:3300/0 conn(0x7fd6e8074b50 0x7fd6e80af7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7fd6e6d9d640 1 -- 172.21.15.57:0/110218354 >> v1:172.21.15.80:6789/0 conn(0x7fd6e8142490 legacy=0x7fd6e81428d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7fd6e559a640 1 -- 172.21.15.57:0/110218354 <== mon.1 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3350878169 0 0) 0x7fd6d8001120 con 0x7fd6e8142490 2024-04-25T23:49:00.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7fd6e6d9d640 1 -- 172.21.15.57:0/110218354 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6e813fdb0 con 0x7fd6e813f3e0 2024-04-25T23:49:00.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7fa156c9f640 1 -- start start 2024-04-25T23:49:00.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7fa156c9f640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa15006f4b0 0x7fa15013bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7fa156c9f640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fa150073430 0x7fa15013e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7fa156c9f640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa150070520 con 0x7fa150070140 2024-04-25T23:49:00.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7fa156c9f640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fa1500708e0 con 0x7fa15006f4b0 2024-04-25T23:49:00.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7fa156c9f640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7fa15013eb80 con 0x7fa150073430 2024-04-25T23:49:00.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7fa14ffff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa15006f4b0 0x7fa15013bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7fa14ffff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa15006f4b0 0x7fa15013bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.57:54174/0 (socket says 172.21.15.57:54174) 2024-04-25T23:49:00.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7fa14ffff640 1 -- 172.21.15.57:0/3126434026 learned_addr learned my addr 172.21.15.57:0/3126434026 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:49:00.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7fa15523c640 1 --2- 172.21.15.57:0/3126434026 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fa150073430 0x7fa15013e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.147+0000 7fa14ffff640 1 -- 172.21.15.57:0/3126434026 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fa150073430 msgr2=0x7fa15013e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:49:00.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.151+0000 7fa14ffff640 1 --2- 172.21.15.57:0/3126434026 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fa150073430 0x7fa15013e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.151+0000 7fa14ffff640 1 -- 172.21.15.57:0/3126434026 >> v1:172.21.15.80:6789/0 conn(0x7fa150070140 legacy=0x7fa15013b7f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T23:49:00.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.151+0000 7fa14ffff640 1 -- 172.21.15.57:0/3126434026 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa15013ed00 con 0x7fa15006f4b0 2024-04-25T23:49:00.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.151+0000 7fa5ec98e640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa5e0037610 0x7fa5e00c7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.151+0000 7fa5ec98e640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa5e0037610 0x7fa5e00c7e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.57:54190/0 (socket says 172.21.15.57:54190) 2024-04-25T23:49:00.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.151+0000 7fa5ec98e640 1 -- 172.21.15.57:0/1153566655 learned_addr learned my addr 172.21.15.57:0/1153566655 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:49:00.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.151+0000 7fd6e6d9d640 1 --2- 172.21.15.57:0/110218354 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fd6e813f3e0 0x7fd6e813f870 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0009d90 tx=0x7fd6e001c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45b5c709ab66b359 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:00.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.151+0000 7f2ac77fe640 1 --2- 172.21.15.57:0/644564634 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f2ac806ee00 0x7f2ac813b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.151+0000 7f2ac77fe640 1 -- 172.21.15.57:0/644564634 >> v1:172.21.15.80:6789/0 conn(0x7f2ac813e780 legacy=0x7f2ac813ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.151+0000 7f2ac77fe640 1 -- 172.21.15.57:0/644564634 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ac806ff70 con 0x7f2ac806f1e0 2024-04-25T23:49:00.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.151+0000 7fa5ed18f640 1 --2- 172.21.15.57:0/1153566655 >> v2:172.21.15.57:3300/0 conn(0x7fa5e002c5e0 0x7fa5e00c1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.151+0000 7f2ac77fe640 1 --2- 172.21.15.57:0/644564634 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ac806f1e0 0x7f2ac813bae0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2abc00d8a0 tx=0x7f2abc01e670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6d581d5fb2bc8c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:00.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.151+0000 7f2ac7fff640 1 --2- 172.21.15.57:0/644564634 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f2ac806ee00 0x7f2ac813b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T23:49:00.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.151+0000 7fd6e559a640 1 -- 172.21.15.57:0/110218354 <== mon.2 v2:172.21.15.57:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e00b2070 con 0x7fd6e813f3e0 2024-04-25T23:49:00.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.151+0000 7fd6e559a640 1 -- 172.21.15.57:0/110218354 <== mon.2 v2:172.21.15.57:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6e00a57c0 con 0x7fd6e813f3e0 2024-04-25T23:49:00.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.151+0000 7fd6e559a640 1 -- 172.21.15.57:0/110218354 <== mon.2 v2:172.21.15.57:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e00a5b90 con 0x7fd6e813f3e0 2024-04-25T23:49:00.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.151+0000 7fa5ec98e640 1 -- 172.21.15.57:0/1153566655 >> v2:172.21.15.57:3300/0 conn(0x7fa5e002c5e0 msgr2=0x7fa5e00c1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:49:00.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.151+0000 7fa5ec98e640 1 --2- 172.21.15.57:0/1153566655 >> v2:172.21.15.57:3300/0 conn(0x7fa5e002c5e0 0x7fa5e00c1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.151+0000 7fa5ec98e640 1 -- 172.21.15.57:0/1153566655 >> v1:172.21.15.57:6789/0 conn(0x7fa5e002b8f0 legacy=0x7fa5e00c1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.151+0000 7fa5ec98e640 1 -- 172.21.15.57:0/1153566655 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5e002b0f0 con 0x7fa5e0037610 2024-04-25T23:49:00.149 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.151+0000 7fa14ffff640 1 --2- 172.21.15.57:0/3126434026 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa15006f4b0 0x7fa15013bf00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa144008870 tx=0x7fa14401ea10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4264f0ead8b1243 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:00.149 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.151+0000 7f1eced85640 1 --2- 172.21.15.57:0/3933796287 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1ec8145c60 0x7f1ec81460d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1ec80b40c0 tx=0x7f1eb801c460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b774571d90381065 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:00.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.151+0000 7fd6e759e640 1 -- 172.21.15.57:0/110218354 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fd6e813f3e0 msgr2=0x7fd6e813f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.151+0000 7fd6e759e640 1 --2- 172.21.15.57:0/110218354 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fd6e813f3e0 0x7fd6e813f870 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0009d90 tx=0x7fd6e001c8c0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.151+0000 7fa14f7fe640 1 -- 172.21.15.57:0/3126434026 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1440ad070 con 0x7fa15006f4b0 2024-04-25T23:49:00.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.151+0000 7fa14f7fe640 1 -- 172.21.15.57:0/3126434026 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1440a79b0 con 0x7fa15006f4b0 2024-04-25T23:49:00.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.151+0000 7fa14f7fe640 1 -- 172.21.15.57:0/3126434026 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1440a7d80 con 0x7fa15006f4b0 2024-04-25T23:49:00.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.151+0000 7f1ecd582640 1 -- 172.21.15.57:0/3933796287 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb80ab620 con 0x7f1ec8145c60 2024-04-25T23:49:00.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.151+0000 7f1ecd582640 1 -- 172.21.15.57:0/3933796287 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1eb80b0070 con 0x7f1ec8145c60 2024-04-25T23:49:00.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.151+0000 7f1ecd582640 1 -- 172.21.15.57:0/3933796287 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb80aba20 con 0x7f1ec8145c60 2024-04-25T23:49:00.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7fa156c9f640 1 -- 172.21.15.57:0/3126434026 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa15006f4b0 msgr2=0x7fa15013bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7fa156c9f640 1 --2- 172.21.15.57:0/3126434026 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa15006f4b0 0x7fa15013bf00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa144008870 tx=0x7fa14401ea10 comp rx=0 tx=0).stop 2024-04-25T23:49:00.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7f2ac6ffd640 1 -- 172.21.15.57:0/644564634 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac00051d0 con 0x7f2ac806f1e0 2024-04-25T23:49:00.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7f2ac6ffd640 1 -- 172.21.15.57:0/644564634 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2abc0b2400 con 0x7f2ac806f1e0 2024-04-25T23:49:00.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7f2ac6ffd640 1 -- 172.21.15.57:0/644564634 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2abc00b040 con 0x7f2ac806f1e0 2024-04-25T23:49:00.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7fd6e759e640 1 -- 172.21.15.57:0/110218354 shutdown_connections 2024-04-25T23:49:00.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7fd6e759e640 1 --2- 172.21.15.57:0/110218354 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fd6e813f3e0 0x7fd6e813f870 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7fd6e759e640 1 --2- 172.21.15.57:0/110218354 >> v2:172.21.15.57:3300/0 conn(0x7fd6e8074b50 0x7fd6e80af7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7fd6e759e640 1 -- 172.21.15.57:0/110218354 >> 172.21.15.57:0/110218354 conn(0x7fd6e80b4dd0 msgr2=0x7fd6e80b51e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:49:00.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7fd6e759e640 1 -- 172.21.15.57:0/110218354 shutdown_connections 2024-04-25T23:49:00.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7fd6e759e640 1 -- 172.21.15.57:0/110218354 wait complete. 2024-04-25T23:49:00.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7fbc09123640 1 --2- 172.21.15.57:0/2282819448 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbbfc035040 0x7fbbfc02d030 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbbf40097c0 tx=0x7fbbf401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=839641d6ed221e05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:00.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7f1ed07e8640 1 -- 172.21.15.57:0/3933796287 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1ec8145c60 msgr2=0x7f1ec81460d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7f1ed07e8640 1 --2- 172.21.15.57:0/3933796287 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1ec8145c60 0x7f1ec81460d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1ec80b40c0 tx=0x7f1eb801c460 comp rx=0 tx=0).stop 2024-04-25T23:49:00.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7fa156c9f640 1 -- 172.21.15.57:0/3126434026 shutdown_connections 2024-04-25T23:49:00.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7fa156c9f640 1 --2- 172.21.15.57:0/3126434026 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fa150073430 0x7fa15013e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7fa156c9f640 1 --2- 172.21.15.57:0/3126434026 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa15006f4b0 0x7fa15013bf00 secure :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa144008870 tx=0x7fa14401ea10 comp rx=0 tx=0).stop 2024-04-25T23:49:00.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7fa156c9f640 1 -- 172.21.15.57:0/3126434026 >> 172.21.15.57:0/3126434026 conn(0x7fa1500b4480 msgr2=0x7fa1500b4890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:49:00.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7fa156c9f640 1 -- 172.21.15.57:0/3126434026 shutdown_connections 2024-04-25T23:49:00.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7f1ed07e8640 1 -- 172.21.15.57:0/3933796287 shutdown_connections 2024-04-25T23:49:00.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7f1ed07e8640 1 --2- 172.21.15.57:0/3933796287 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1ec8145c60 0x7f1ec81460d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7f1ed07e8640 1 --2- 172.21.15.57:0/3933796287 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f1ec80a9e50 0x7f1ec813f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7f1ed07e8640 1 --2- 172.21.15.57:0/3933796287 >> v2:172.21.15.57:3300/0 conn(0x7f1ec80b51e0 0x7f1ec813f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7f1ed07e8640 1 -- 172.21.15.57:0/3933796287 >> 172.21.15.57:0/3933796287 conn(0x7f1ec8074af0 msgr2=0x7f1ec8074f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:49:00.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7f1ed07e8640 1 -- 172.21.15.57:0/3933796287 shutdown_connections 2024-04-25T23:49:00.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7f2acd379640 1 -- 172.21.15.57:0/644564634 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ac806f1e0 msgr2=0x7f2ac813bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7f2acd379640 1 --2- 172.21.15.57:0/644564634 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ac806f1e0 0x7f2ac813bae0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2abc00d8a0 tx=0x7f2abc01e670 comp rx=0 tx=0).stop 2024-04-25T23:49:00.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7f2acd379640 1 -- 172.21.15.57:0/644564634 shutdown_connections 2024-04-25T23:49:00.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7f2acd379640 1 --2- 172.21.15.57:0/644564634 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ac806f1e0 0x7f2ac813bae0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7f2acd379640 1 --2- 172.21.15.57:0/644564634 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f2ac806ee00 0x7f2ac813b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7f2acd379640 1 -- 172.21.15.57:0/644564634 >> 172.21.15.57:0/644564634 conn(0x7f2ac80752a0 msgr2=0x7f2ac80b3e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:49:00.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7f2acd379640 1 -- 172.21.15.57:0/644564634 shutdown_connections 2024-04-25T23:49:00.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7fd6e759e640 1 Processor -- start 2024-04-25T23:49:00.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7fd6e759e640 1 -- start start 2024-04-25T23:49:00.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7fa156c9f640 1 -- 172.21.15.57:0/3126434026 wait complete. 2024-04-25T23:49:00.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7fa156c9f640 1 Processor -- start 2024-04-25T23:49:00.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7fa5ec98e640 1 --2- 172.21.15.57:0/1153566655 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5e0037610 0x7fa5e00c7e20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa5e002a290 tx=0x7fa5dc01eb60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a944926a04c14b75 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:00.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7fa5e77fe640 1 -- 172.21.15.57:0/1153566655 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc0b4070 con 0x7fa5e0037610 2024-04-25T23:49:00.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7f1ed07e8640 1 -- 172.21.15.57:0/3933796287 wait complete. 2024-04-25T23:49:00.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7f1ed07e8640 1 Processor -- start 2024-04-25T23:49:00.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7fa5e77fe640 1 -- 172.21.15.57:0/1153566655 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5dc0a7940 con 0x7fa5e0037610 2024-04-25T23:49:00.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7f1ed07e8640 1 -- start start 2024-04-25T23:49:00.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7fbc08922640 1 -- 172.21.15.57:0/2282819448 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf40aa600 con 0x7fbbfc035040 2024-04-25T23:49:00.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7fbc08922640 1 -- 172.21.15.57:0/2282819448 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbf40af070 con 0x7fbbfc035040 2024-04-25T23:49:00.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7fbc08922640 1 -- 172.21.15.57:0/2282819448 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf40aa9c0 con 0x7fbbfc035040 2024-04-25T23:49:00.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7fd6e759e640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fd6e8074b50 0x7fd6e813b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7fd6e759e640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd6e813f3e0 0x7fd6e8137600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7fd6e759e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd6e8142490 0x7fd6e80b4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7fd6e759e640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e814f950 con 0x7fd6e813f3e0 2024-04-25T23:49:00.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7fd6e759e640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e814f7d0 con 0x7fd6e8142490 2024-04-25T23:49:00.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7fd6e759e640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7fd6e814fad0 con 0x7fd6e8074b50 2024-04-25T23:49:00.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7fa156c9f640 1 -- start start 2024-04-25T23:49:00.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7fa156c9f640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fa150070140 0x7fa15013cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7fa156c9f640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa150073430 0x7fa15013dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7fa156c9f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa150139800 0x7fa150139c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7fa156c9f640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fa15014bac0 con 0x7fa150073430 2024-04-25T23:49:00.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7fa156c9f640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa15014b940 con 0x7fa150139800 2024-04-25T23:49:00.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7fa156c9f640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7fa15014bc40 con 0x7fa150070140 2024-04-25T23:49:00.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7f2acd379640 1 -- 172.21.15.57:0/644564634 wait complete. 2024-04-25T23:49:00.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7f2acd379640 1 Processor -- start 2024-04-25T23:49:00.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7fd6e6d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd6e8142490 0x7fd6e80b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7fd6e6d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd6e8142490 0x7fd6e80b4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.57:54212/0 (socket says 172.21.15.57:54212) 2024-04-25T23:49:00.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7fd6e6d9d640 1 -- 172.21.15.57:0/2247819301 learned_addr learned my addr 172.21.15.57:0/2247819301 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:49:00.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7fd6e5d9b640 1 --2- 172.21.15.57:0/2247819301 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd6e813f3e0 0x7fd6e8137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7f1ed07e8640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1ec80a9e50 0x7f1ec80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7fd6e659c640 1 --2- 172.21.15.57:0/2247819301 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fd6e8074b50 0x7fd6e813b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7fa15523c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa150139800 0x7fa150139c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7fa154a3b640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fa150070140 0x7fa15013cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7fa154a3b640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fa150070140 0x7fa15013cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3301/0 says I am v2:172.21.15.57:47282/0 (socket says 172.21.15.57:47282) 2024-04-25T23:49:00.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7fa15523c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa150139800 0x7fa150139c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.57:54202/0 (socket says 172.21.15.57:54202) 2024-04-25T23:49:00.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7fbc0a926640 1 -- 172.21.15.57:0/2282819448 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbbfc035040 msgr2=0x7fbbfc02d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7fbc0a926640 1 --2- 172.21.15.57:0/2282819448 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbbfc035040 0x7fbbfc02d030 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbbf40097c0 tx=0x7fbbf401c4a0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7f1ed07e8640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f1ec80b51e0 0x7f1ec81422c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7f2acd379640 1 -- start start 2024-04-25T23:49:00.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7fbc0a926640 1 -- 172.21.15.57:0/2282819448 shutdown_connections 2024-04-25T23:49:00.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7fbc0a926640 1 --2- 172.21.15.57:0/2282819448 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbbfc035040 0x7fbbfc02d030 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7fbc0a926640 1 -- 172.21.15.57:0/2282819448 >> 172.21.15.57:0/2282819448 conn(0x7fbbfc0063f0 msgr2=0x7fbbfc006800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:49:00.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7fd6e6d9d640 1 -- 172.21.15.57:0/2247819301 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fd6e8074b50 msgr2=0x7fd6e813b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7fd6e6d9d640 1 --2- 172.21.15.57:0/2247819301 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fd6e8074b50 0x7fd6e813b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7fd6e6d9d640 1 -- 172.21.15.57:0/2247819301 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd6e813f3e0 msgr2=0x7fd6e8137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7fd6e6d9d640 1 --2- 172.21.15.57:0/2247819301 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd6e813f3e0 0x7fd6e8137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7fa5ed990640 1 -- 172.21.15.57:0/1153566655 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5e0037610 msgr2=0x7fa5e00c7e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7fa5ed990640 1 --2- 172.21.15.57:0/1153566655 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5e0037610 0x7fa5e00c7e20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa5e002a290 tx=0x7fa5dc01eb60 comp rx=0 tx=0).stop 2024-04-25T23:49:00.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7fd6e6d9d640 1 -- 172.21.15.57:0/2247819301 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6e813aef0 con 0x7fd6e8142490 2024-04-25T23:49:00.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7fbc0a926640 1 -- 172.21.15.57:0/2282819448 shutdown_connections 2024-04-25T23:49:00.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7fbc0a926640 1 -- 172.21.15.57:0/2282819448 wait complete. 2024-04-25T23:49:00.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7fa5ed990640 1 -- 172.21.15.57:0/1153566655 shutdown_connections 2024-04-25T23:49:00.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7fa5ed990640 1 --2- 172.21.15.57:0/1153566655 >> v2:172.21.15.57:3300/0 conn(0x7fa5e002c5e0 0x7fa5e00c1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7fa5ed990640 1 --2- 172.21.15.57:0/1153566655 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5e0037610 0x7fa5e00c7e20 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7fa5ed990640 1 -- 172.21.15.57:0/1153566655 >> 172.21.15.57:0/1153566655 conn(0x7fa5e003a8d0 msgr2=0x7fa5e0036bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:49:00.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7f1ecdd83640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f1ec80b51e0 0x7f1ec81422c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7f1ed07e8640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ec8145c60 0x7f1ec80b8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7fd6e6d9d640 1 --2- 172.21.15.57:0/2247819301 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd6e8142490 0x7fd6e80b4330 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd6e001a4f0 tx=0x7fd6e0019660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:00.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7f2acd379640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f2ac806ee00 0x7f2ac8136310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7f1ed07e8640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec80b3700 con 0x7f1ec8145c60 2024-04-25T23:49:00.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7f1ed07e8640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec80b3580 con 0x7f1ec80a9e50 2024-04-25T23:49:00.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7f1ed07e8640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f1ec80b3880 con 0x7f1ec80b51e0 2024-04-25T23:49:00.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7f2acd379640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ac806f1e0 0x7f2ac80b1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7f1ece584640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1ec80a9e50 0x7f1ec80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7f1ece584640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1ec80a9e50 0x7f1ec80b77a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.57:54216/0 (socket says 172.21.15.57:54216) 2024-04-25T23:49:00.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7f1ecdd83640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f1ec80b51e0 0x7f1ec81422c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3301/0 says I am v2:172.21.15.57:47290/0 (socket says 172.21.15.57:47290) 2024-04-25T23:49:00.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7f1ece584640 1 -- 172.21.15.57:0/713409486 learned_addr learned my addr 172.21.15.57:0/713409486 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:49:00.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7fd6d6ffd640 1 -- 172.21.15.57:0/2247819301 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e00bd020 con 0x7fd6e8142490 2024-04-25T23:49:00.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7fd6d6ffd640 1 -- 172.21.15.57:0/2247819301 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6e00b3040 con 0x7fd6e8142490 2024-04-25T23:49:00.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7fd6e759e640 1 -- 172.21.15.57:0/2247819301 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6e813b1d0 con 0x7fd6e8142490 2024-04-25T23:49:00.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7f2ac7fff640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f2ac806ee00 0x7f2ac8136310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7f2ac7fff640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f2ac806ee00 0x7f2ac8136310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3301/0 says I am v2:172.21.15.57:47304/0 (socket says 172.21.15.57:47304) 2024-04-25T23:49:00.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7f2ac77fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ac806f1e0 0x7f2ac80b1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7f2ac7fff640 1 -- 172.21.15.57:0/795898583 learned_addr learned my addr 172.21.15.57:0/795898583 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:49:00.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7f2acd379640 1 --2- 172.21.15.57:0/795898583 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ac813e780 0x7f2ac80b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7f1ece584640 1 -- 172.21.15.57:0/713409486 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f1ec80b51e0 msgr2=0x7f1ec81422c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7f1ece584640 1 --2- 172.21.15.57:0/713409486 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f1ec80b51e0 0x7f1ec81422c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7f1ece584640 1 -- 172.21.15.57:0/713409486 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ec8145c60 msgr2=0x7f1ec80b8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7f1ece584640 1 --2- 172.21.15.57:0/713409486 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ec8145c60 0x7f1ec80b8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7f1ece584640 1 -- 172.21.15.57:0/713409486 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ec8138f60 con 0x7f1ec80a9e50 2024-04-25T23:49:00.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7f1ece584640 1 --2- 172.21.15.57:0/713409486 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1ec80a9e50 0x7f1ec80b77a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1ec400b550 tx=0x7f1ec400ba10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:00.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7fd6d6ffd640 1 -- 172.21.15.57:0/2247819301 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e00b2070 con 0x7fd6e8142490 2024-04-25T23:49:00.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7fd6e759e640 1 -- 172.21.15.57:0/2247819301 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6e8139560 con 0x7fd6e8142490 2024-04-25T23:49:00.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7f2accb78640 1 --2- 172.21.15.57:0/795898583 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ac813e780 0x7f2ac80b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.155+0000 7fa154a3b640 1 -- 172.21.15.57:0/1105575273 learned_addr learned my addr 172.21.15.57:0/1105575273 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:49:00.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7fa5ed990640 1 -- 172.21.15.57:0/1153566655 shutdown_connections 2024-04-25T23:49:00.176 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7fa14ffff640 1 --2- 172.21.15.57:0/1105575273 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa150073430 0x7fa15013dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.176 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7fbc0a926640 1 Processor -- start 2024-04-25T23:49:00.176 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7f1ebeffd640 1 -- 172.21.15.57:0/713409486 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec4014070 con 0x7f1ec80a9e50 2024-04-25T23:49:00.176 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7f1ebeffd640 1 -- 172.21.15.57:0/713409486 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ec4010870 con 0x7f1ec80a9e50 2024-04-25T23:49:00.176 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7f1ebeffd640 1 -- 172.21.15.57:0/713409486 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec4010c40 con 0x7f1ec80a9e50 2024-04-25T23:49:00.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7fd6e759e640 1 -- 172.21.15.57:0/2247819301 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd6cc000f90 con 0x7fd6e8142490 2024-04-25T23:49:00.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7fa5ed990640 1 -- 172.21.15.57:0/1153566655 wait complete. 2024-04-25T23:49:00.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7fa154a3b640 1 -- 172.21.15.57:0/1105575273 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa150139800 msgr2=0x7fa150139c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7fa154a3b640 1 --2- 172.21.15.57:0/1105575273 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa150139800 0x7fa150139c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7fa154a3b640 1 -- 172.21.15.57:0/1105575273 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa150073430 msgr2=0x7fa15013dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7fa154a3b640 1 --2- 172.21.15.57:0/1105575273 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa150073430 0x7fa15013dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7fa154a3b640 1 -- 172.21.15.57:0/1105575273 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa15013e600 con 0x7fa150070140 2024-04-25T23:49:00.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7fbc0a926640 1 -- start start 2024-04-25T23:49:00.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7fbc0a926640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbbfc02d590 0x7fbbfc0c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7fbc0a926640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fbbfc035040 0x7fbbfc0bf500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7fbc0a926640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbbfc036e30 0x7fbbfc0c5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7fbc0a926640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fbbfc008010 con 0x7fbbfc02d590 2024-04-25T23:49:00.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7fbc0a926640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fbbfc007e90 con 0x7fbbfc036e30 2024-04-25T23:49:00.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7fbc0a926640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7fbbfc008190 con 0x7fbbfc035040 2024-04-25T23:49:00.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7f1ed07e8640 1 -- 172.21.15.57:0/713409486 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ec8139240 con 0x7f1ec80a9e50 2024-04-25T23:49:00.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7f1ed07e8640 1 -- 172.21.15.57:0/713409486 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ec80b86b0 con 0x7f1ec80a9e50 2024-04-25T23:49:00.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7f2acd379640 1 -- 172.21.15.57:0/795898583 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac814b860 con 0x7f2ac813e780 2024-04-25T23:49:00.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7f2acd379640 1 -- 172.21.15.57:0/795898583 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac814b6e0 con 0x7f2ac806f1e0 2024-04-25T23:49:00.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7f2acd379640 1 -- 172.21.15.57:0/795898583 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f2ac814b9e0 con 0x7f2ac806ee00 2024-04-25T23:49:00.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7f1ed07e8640 1 -- 172.21.15.57:0/713409486 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1eb0000f90 con 0x7f1ec80a9e50 2024-04-25T23:49:00.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7fbc09123640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fbbfc035040 0x7fbbfc0bf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7fbc09123640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fbbfc035040 0x7fbbfc0bf500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3301/0 says I am v2:172.21.15.57:47314/0 (socket says 172.21.15.57:47314) 2024-04-25T23:49:00.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7fbc09123640 1 -- 172.21.15.57:0/2020037778 learned_addr learned my addr 172.21.15.57:0/2020037778 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:49:00.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7fbc0a125640 1 --2- 172.21.15.57:0/2020037778 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbbfc036e30 0x7fbbfc0c5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7fbc09924640 1 --2- 172.21.15.57:0/2020037778 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbbfc02d590 0x7fbbfc0c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7fbc0a125640 1 -- 172.21.15.57:0/2020037778 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fbbfc035040 msgr2=0x7fbbfc0bf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7fbc0a125640 1 --2- 172.21.15.57:0/2020037778 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fbbfc035040 0x7fbbfc0bf500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7fbc0a125640 1 -- 172.21.15.57:0/2020037778 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbbfc02d590 msgr2=0x7fbbfc0c83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7fbc0a125640 1 --2- 172.21.15.57:0/2020037778 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbbfc02d590 0x7fbbfc0c83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7fbc0a125640 1 -- 172.21.15.57:0/2020037778 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbfc0c4780 con 0x7fbbfc036e30 2024-04-25T23:49:00.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7fbc09123640 1 --2- 172.21.15.57:0/2020037778 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fbbfc035040 0x7fbbfc0bf500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T23:49:00.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.163+0000 7f2ac77fe640 1 -- 172.21.15.57:0/795898583 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f2ac806ee00 msgr2=0x7f2ac8136310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.163+0000 7f2ac77fe640 1 --2- 172.21.15.57:0/795898583 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f2ac806ee00 0x7f2ac8136310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.163+0000 7f2ac77fe640 1 -- 172.21.15.57:0/795898583 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ac813e780 msgr2=0x7f2ac80b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.163+0000 7f2ac77fe640 1 --2- 172.21.15.57:0/795898583 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ac813e780 0x7f2ac80b2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.163+0000 7f2ac77fe640 1 -- 172.21.15.57:0/795898583 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ac80b26b0 con 0x7f2ac806f1e0 2024-04-25T23:49:00.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.163+0000 7fbc0a125640 1 --2- 172.21.15.57:0/2020037778 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbbfc036e30 0x7fbbfc0c5ee0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbc040689b0 tx=0x7fbc040685e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:00.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.163+0000 7fd6d6ffd640 1 -- 172.21.15.57:0/2247819301 <== mon.1 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 4) ==== 87354+0+0 (secure 0 0 0) 0x7fd6e00c0020 con 0x7fd6e8142490 2024-04-25T23:49:00.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.163+0000 7fd6d6ffd640 1 --2- 172.21.15.57:0/2247819301 >> v2:172.21.15.80:6816/2493457677 conn(0x7fd6d0067940 0x7fd6d0069e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.163+0000 7fd6d6ffd640 1 -- 172.21.15.57:0/2247819301 <== mon.1 v2:172.21.15.80:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4090+0+0 (secure 0 0 0) 0x7fd6e0120ee0 con 0x7fd6e8142490 2024-04-25T23:49:00.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.163+0000 7fd6d6ffd640 1 --2- 172.21.15.57:0/2247819301 >> v2:172.21.15.80:6800/4069821322 conn(0x7fd6d006d920 0x7fd6d006fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.163+0000 7fd6d6ffd640 1 -- 172.21.15.57:0/2247819301 --> v2:172.21.15.80:6800/4069821322 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd6e00a6e50 con 0x7fd6d006d920 2024-04-25T23:49:00.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.163+0000 7fd6d6ffd640 1 -- 172.21.15.57:0/2247819301 <== mon.1 v2:172.21.15.80:3300/0 6 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fd6e0121280 con 0x7fd6e8142490 2024-04-25T23:49:00.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.163+0000 7fd6e5d9b640 1 --2- 172.21.15.57:0/2247819301 >> v2:172.21.15.80:6800/4069821322 conn(0x7fd6d006d920 0x7fd6d006fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7fa154a3b640 1 --2- 172.21.15.57:0/1105575273 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fa150070140 0x7fa15013cdb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa13c00b4a0 tx=0x7fa13c00b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:00.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7fa14d7fa640 1 -- 172.21.15.57:0/1105575273 <== mon.2 v2:172.21.15.57:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa13c014070 con 0x7fa150070140 2024-04-25T23:49:00.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7fa14d7fa640 1 -- 172.21.15.57:0/1105575273 <== mon.2 v2:172.21.15.57:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa13c010610 con 0x7fa150070140 2024-04-25T23:49:00.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7fa14d7fa640 1 -- 172.21.15.57:0/1105575273 <== mon.2 v2:172.21.15.57:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa13c010a20 con 0x7fa150070140 2024-04-25T23:49:00.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.163+0000 7fa156c9f640 1 -- 172.21.15.57:0/1105575273 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa15013e010 con 0x7fa150070140 2024-04-25T23:49:00.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.163+0000 7fa156c9f640 1 -- 172.21.15.57:0/1105575273 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa15013a390 con 0x7fa150070140 2024-04-25T23:49:00.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.163+0000 7fa156c9f640 1 -- 172.21.15.57:0/1105575273 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa15013aae0 con 0x7fa150070140 2024-04-25T23:49:00.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7fa5ed990640 1 Processor -- start 2024-04-25T23:49:00.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.159+0000 7fa5ed990640 1 -- start start 2024-04-25T23:49:00.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.163+0000 7fa5ed990640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa5e002b8f0 0x7fa5e00c4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.163+0000 7fa5ed990640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fa5e002c5e0 0x7fa5e002ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.163+0000 7fa5ed990640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5e0037610 0x7fa5e0163470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.163+0000 7fa5ed990640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e00076d0 con 0x7fa5e002b8f0 2024-04-25T23:49:00.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.163+0000 7f2ac77fe640 1 --2- 172.21.15.57:0/795898583 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ac806f1e0 0x7f2ac80b1c30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2abc0ae000 tx=0x7f2abc01b410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:00.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.163+0000 7fa5ed990640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e0007550 con 0x7fa5e0037610 2024-04-25T23:49:00.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.163+0000 7fa5ed990640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7fa5e0007850 con 0x7fa5e002c5e0 2024-04-25T23:49:00.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.163+0000 7fa5ed18f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5e0037610 0x7fa5e0163470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.163+0000 7fa5ed18f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5e0037610 0x7fa5e0163470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.57:54234/0 (socket says 172.21.15.57:54234) 2024-04-25T23:49:00.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.163+0000 7fa5ed18f640 1 -- 172.21.15.57:0/3726924551 learned_addr learned my addr 172.21.15.57:0/3726924551 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:49:00.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.163+0000 7fa5e7fff640 1 --2- 172.21.15.57:0/3726924551 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fa5e002c5e0 0x7fa5e002ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.163+0000 7fa5ed18f640 1 -- 172.21.15.57:0/3726924551 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fa5e002c5e0 msgr2=0x7fa5e002ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.163+0000 7fa5ed18f640 1 --2- 172.21.15.57:0/3726924551 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fa5e002c5e0 0x7fa5e002ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.163+0000 7fa5ed18f640 1 -- 172.21.15.57:0/3726924551 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa5e002b8f0 msgr2=0x7fa5e00c4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.163+0000 7fa5ed18f640 1 --2- 172.21.15.57:0/3726924551 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa5e002b8f0 0x7fa5e00c4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.163+0000 7fa5ed18f640 1 -- 172.21.15.57:0/3726924551 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5e0031b60 con 0x7fa5e0037610 2024-04-25T23:49:00.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.163+0000 7fc55d9ea640 1 Processor -- start 2024-04-25T23:49:00.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.163+0000 7fbbfa7fc640 1 -- 172.21.15.57:0/2020037778 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc0408f070 con 0x7fbbfc036e30 2024-04-25T23:49:00.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.163+0000 7fbbfa7fc640 1 -- 172.21.15.57:0/2020037778 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc04073820 con 0x7fbbfc036e30 2024-04-25T23:49:00.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.163+0000 7fbc0a926640 1 -- 172.21.15.57:0/2020037778 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbfc0bfbd0 con 0x7fbbfc036e30 2024-04-25T23:49:00.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.163+0000 7fbc0a926640 1 -- 172.21.15.57:0/2020037778 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbfc0c6420 con 0x7fbbfc036e30 2024-04-25T23:49:00.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.163+0000 7fbbfa7fc640 1 -- 172.21.15.57:0/2020037778 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc040935f0 con 0x7fbbfc036e30 2024-04-25T23:49:00.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.163+0000 7fa5ed18f640 1 --2- 172.21.15.57:0/3726924551 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5e0037610 0x7fa5e0163470 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa5e802d710 tx=0x7fa5e80685e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:00.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.163+0000 7f1ebeffd640 1 -- 172.21.15.57:0/713409486 <== mon.1 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 4) ==== 87354+0+0 (secure 0 0 0) 0x7f1ec4017070 con 0x7f1ec80a9e50 2024-04-25T23:49:00.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.163+0000 7f1ebeffd640 1 --2- 172.21.15.57:0/713409486 >> v2:172.21.15.80:6816/2493457677 conn(0x7f1eac0678f0 0x7f1eac069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.163+0000 7f1ebeffd640 1 -- 172.21.15.57:0/713409486 <== mon.1 v2:172.21.15.80:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4090+0+0 (secure 0 0 0) 0x7f1ec4085790 con 0x7f1ec80a9e50 2024-04-25T23:49:00.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.163+0000 7f1ebeffd640 1 --2- 172.21.15.57:0/713409486 >> v2:172.21.15.57:6800/1899803014 conn(0x7f1eac06d8d0 0x7f1eac06fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.163+0000 7f1ebeffd640 1 -- 172.21.15.57:0/713409486 --> v2:172.21.15.57:6800/1899803014 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1eac070360 con 0x7f1eac06d8d0 2024-04-25T23:49:00.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.163+0000 7f1ebeffd640 1 -- 172.21.15.57:0/713409486 <== mon.1 v2:172.21.15.80:3300/0 6 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f1ec4085b30 con 0x7f1ec80a9e50 2024-04-25T23:49:00.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.163+0000 7f1eced85640 1 --2- 172.21.15.57:0/713409486 >> v2:172.21.15.57:6800/1899803014 conn(0x7f1eac06d8d0 0x7f1eac06fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.163+0000 7f1eced85640 1 --2- 172.21.15.57:0/713409486 >> v2:172.21.15.57:6800/1899803014 conn(0x7f1eac06d8d0 0x7f1eac06fcf0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:00.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.163+0000 7fd6e5d9b640 1 --2- 172.21.15.57:0/2247819301 >> v2:172.21.15.80:6800/4069821322 conn(0x7fd6d006d920 0x7fd6d006fd40 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:00.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.163+0000 7fc55d9ea640 1 -- start start 2024-04-25T23:49:00.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.163+0000 7f2ac4ff9640 1 -- 172.21.15.57:0/795898583 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2abc004230 con 0x7f2ac806f1e0 2024-04-25T23:49:00.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.163+0000 7f2ac4ff9640 1 -- 172.21.15.57:0/795898583 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2abc0043d0 con 0x7f2ac806f1e0 2024-04-25T23:49:00.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.163+0000 7f2ac4ff9640 1 -- 172.21.15.57:0/795898583 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2abc003840 con 0x7f2ac806f1e0 2024-04-25T23:49:00.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.163+0000 7f2acd379640 1 -- 172.21.15.57:0/795898583 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ac80b3b40 con 0x7f2ac806f1e0 2024-04-25T23:49:00.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.163+0000 7f2acd379640 1 -- 172.21.15.57:0/795898583 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ac80b3340 con 0x7f2ac806f1e0 2024-04-25T23:49:00.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.163+0000 7fa5e57fa640 1 -- 172.21.15.57:0/3726924551 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e8083670 con 0x7fa5e0037610 2024-04-25T23:49:00.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.163+0000 7fa5e57fa640 1 -- 172.21.15.57:0/3726924551 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5e807e070 con 0x7fa5e0037610 2024-04-25T23:49:00.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.163+0000 7fa5e57fa640 1 -- 172.21.15.57:0/3726924551 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e8083a50 con 0x7fa5e0037610 2024-04-25T23:49:00.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.163+0000 7fd6d6ffd640 1 -- 172.21.15.57:0/2247819301 <== osd.4 v2:172.21.15.80:6800/4069821322 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd6e00a6e50 con 0x7fd6d006d920 2024-04-25T23:49:00.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.163+0000 7fa5ed990640 1 -- 172.21.15.57:0/3726924551 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5e002f770 con 0x7fa5e0037610 2024-04-25T23:49:00.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.163+0000 7fa5ed990640 1 -- 172.21.15.57:0/3726924551 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5e0030490 con 0x7fa5e0037610 2024-04-25T23:49:00.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.163+0000 7f1ebeffd640 1 -- 172.21.15.57:0/713409486 <== osd.3 v2:172.21.15.57:6800/1899803014 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1eac070360 con 0x7f1eac06d8d0 2024-04-25T23:49:00.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.167+0000 7fbbfa7fc640 1 -- 172.21.15.57:0/2020037778 <== mon.1 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 4) ==== 87354+0+0 (secure 0 0 0) 0x7fbc04092070 con 0x7fbbfc036e30 2024-04-25T23:49:00.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.167+0000 7fc55d9ea640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc5500c0ea0 0x7fc5500c12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.167+0000 7fc55d9ea640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fc5500cc4a0 0x7fc5500ce890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.167+0000 7fc55d9ea640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc550033440 con 0x7fc550034cb0 2024-04-25T23:49:00.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.167+0000 7fc55d9ea640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fc550031560 con 0x7fc5500c0ea0 2024-04-25T23:49:00.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.167+0000 7fbbfa7fc640 1 --2- 172.21.15.57:0/2020037778 >> v2:172.21.15.80:6816/2493457677 conn(0x7fbbe80678c0 0x7fbbe8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.167+0000 7fc55d9ea640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7fc5500c1820 con 0x7fc5500cc4a0 2024-04-25T23:49:00.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.167+0000 7fa5ed990640 1 -- 172.21.15.57:0/3726924551 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa5c8000f90 con 0x7fa5e0037610 2024-04-25T23:49:00.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.167+0000 7fa14d7fa640 1 -- 172.21.15.57:0/1105575273 <== mon.2 v2:172.21.15.57:3301/0 4 ==== mgrmap(e 4) ==== 87354+0+0 (secure 0 0 0) 0x7fa13c017070 con 0x7fa150070140 2024-04-25T23:49:00.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.167+0000 7fa14d7fa640 1 --2- 172.21.15.57:0/1105575273 >> v2:172.21.15.80:6816/2493457677 conn(0x7fa134067820 0x7fa134069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.167+0000 7fa14d7fa640 1 -- 172.21.15.57:0/1105575273 <== mon.2 v2:172.21.15.57:3301/0 5 ==== osd_map(17..17 src has 1..17) ==== 4090+0+0 (secure 0 0 0) 0x7fa13c085460 con 0x7fa150070140 2024-04-25T23:49:00.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.167+0000 7fbbfa7fc640 1 -- 172.21.15.57:0/2020037778 <== mon.1 v2:172.21.15.80:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4090+0+0 (secure 0 0 0) 0x7fbc04100980 con 0x7fbbfc036e30 2024-04-25T23:49:00.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.167+0000 7fa14d7fa640 1 --2- 172.21.15.57:0/1105575273 >> v2:172.21.15.80:6811/2954555099 conn(0x7fa13406d800 0x7fa13406fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.167+0000 7fa14d7fa640 1 -- 172.21.15.57:0/1105575273 --> v2:172.21.15.80:6811/2954555099 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa134070290 con 0x7fa13406d800 2024-04-25T23:49:00.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.167+0000 7fa14d7fa640 1 -- 172.21.15.57:0/1105575273 <== mon.2 v2:172.21.15.57:3301/0 6 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fa13c085800 con 0x7fa150070140 2024-04-25T23:49:00.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.167+0000 7fbc0a926640 1 --2- 172.21.15.57:0/2020037778 >> v2:172.21.15.57:6808/2225240106 conn(0x7fbbec001640 0x7fbbec003b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.167+0000 7fbc0a926640 1 -- 172.21.15.57:0/2020037778 --> v2:172.21.15.57:6808/2225240106 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbbec006ab0 con 0x7fbbec001640 2024-04-25T23:49:00.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.167+0000 7fbc09123640 1 --2- 172.21.15.57:0/2020037778 >> v2:172.21.15.57:6808/2225240106 conn(0x7fbbec001640 0x7fbbec003b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.167+0000 7fa15523c640 1 --2- 172.21.15.57:0/1105575273 >> v2:172.21.15.80:6811/2954555099 conn(0x7fa13406d800 0x7fa13406fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.167+0000 7fc5577fe640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fc5500cc4a0 0x7fc5500ce890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.167+0000 7fc5577fe640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fc5500cc4a0 0x7fc5500ce890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3301/0 says I am v2:172.21.15.57:47344/0 (socket says 172.21.15.57:47344) 2024-04-25T23:49:00.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.167+0000 7fc5577fe640 1 -- 172.21.15.57:0/18981222 learned_addr learned my addr 172.21.15.57:0/18981222 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:49:00.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.167+0000 7f2ac4ff9640 1 -- 172.21.15.57:0/795898583 <== mon.1 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 4) ==== 87354+0+0 (secure 0 0 0) 0x7f2abc0039e0 con 0x7f2ac806f1e0 2024-04-25T23:49:00.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.167+0000 7f2ac4ff9640 1 --2- 172.21.15.57:0/795898583 >> v2:172.21.15.80:6816/2493457677 conn(0x7f2ab40678c0 0x7f2ab4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.167+0000 7f2ac4ff9640 1 -- 172.21.15.57:0/795898583 <== mon.1 v2:172.21.15.80:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4090+0+0 (secure 0 0 0) 0x7f2abc11fb70 con 0x7f2ac806f1e0 2024-04-25T23:49:00.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.167+0000 7fbc09123640 1 --2- 172.21.15.57:0/2020037778 >> v2:172.21.15.57:6808/2225240106 conn(0x7fbbec001640 0x7fbbec003b00 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:00.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.167+0000 7fc5577fe640 1 -- 172.21.15.57:0/18981222 >> v2:172.21.15.80:3300/0 conn(0x7fc5500c0ea0 msgr2=0x7fc5500c12e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T23:49:00.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.167+0000 7fc5577fe640 1 --2- 172.21.15.57:0/18981222 >> v2:172.21.15.80:3300/0 conn(0x7fc5500c0ea0 0x7fc5500c12e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.167+0000 7fc5577fe640 1 -- 172.21.15.57:0/18981222 >> v1:172.21.15.57:6789/0 conn(0x7fc550034cb0 legacy=0x7fc5500c3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:49:00.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.167+0000 7fc5577fe640 1 -- 172.21.15.57:0/18981222 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc550032800 con 0x7fc5500cc4a0 2024-04-25T23:49:00.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.167+0000 7fbc09924640 1 --2- 172.21.15.57:0/2020037778 >> v2:172.21.15.80:6816/2493457677 conn(0x7fbbe80678c0 0x7fbbe8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.167+0000 7fa5e57fa640 1 -- 172.21.15.57:0/3726924551 <== mon.1 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 4) ==== 87354+0+0 (secure 0 0 0) 0x7fa5e8083bf0 con 0x7fa5e0037610 2024-04-25T23:49:00.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.167+0000 7fa5e57fa640 1 --2- 172.21.15.57:0/3726924551 >> v2:172.21.15.80:6816/2493457677 conn(0x7fa5cc0678c0 0x7fa5cc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.167+0000 7fa5e57fa640 1 -- 172.21.15.57:0/3726924551 <== mon.1 v2:172.21.15.80:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4090+0+0 (secure 0 0 0) 0x7fa5e81119a0 con 0x7fa5e0037610 2024-04-25T23:49:00.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.167+0000 7fa5e57fa640 1 --2- 172.21.15.57:0/3726924551 >> v2:172.21.15.57:6804/1976583347 conn(0x7fa5cc06d8a0 0x7fa5cc06fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.167+0000 7fa5e57fa640 1 -- 172.21.15.57:0/3726924551 --> v2:172.21.15.57:6804/1976583347 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa5e8084e50 con 0x7fa5cc06d8a0 2024-04-25T23:49:00.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.167+0000 7fa5e7fff640 1 --2- 172.21.15.57:0/3726924551 >> v2:172.21.15.57:6804/1976583347 conn(0x7fa5cc06d8a0 0x7fa5cc06fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.167+0000 7f1ecdd83640 1 --2- 172.21.15.57:0/713409486 >> v2:172.21.15.80:6816/2493457677 conn(0x7f1eac0678f0 0x7f1eac069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.171+0000 7f2ac7fff640 1 --2- 172.21.15.57:0/795898583 >> v2:172.21.15.80:6816/2493457677 conn(0x7f2ab40678c0 0x7f2ab4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.171+0000 7fbbfa7fc640 1 -- 172.21.15.57:0/2020037778 <== osd.1 v2:172.21.15.57:6808/2225240106 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbbec006ab0 con 0x7fbbec001640 2024-04-25T23:49:00.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.171+0000 7fbc09924640 1 --2- 172.21.15.57:0/2020037778 >> v2:172.21.15.80:6816/2493457677 conn(0x7fbbe80678c0 0x7fbbe8069d80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbc00007fd0 tx=0x7fbc00007d80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:00.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.171+0000 7fa5ec98e640 1 --2- 172.21.15.57:0/3726924551 >> v2:172.21.15.80:6816/2493457677 conn(0x7fa5cc0678c0 0x7fa5cc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.171+0000 7fa5e57fa640 1 -- 172.21.15.57:0/3726924551 <== mon.1 v2:172.21.15.80:3300/0 6 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fa5e80e1ee0 con 0x7fa5e0037610 2024-04-25T23:49:00.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.171+0000 7fd6e659c640 1 --2- 172.21.15.57:0/2247819301 >> v2:172.21.15.80:6816/2493457677 conn(0x7fd6d0067940 0x7fd6d0069e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.167+0000 7f0bd759e640 1 Processor -- start 2024-04-25T23:49:00.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.167+0000 7f0bd759e640 1 -- start start 2024-04-25T23:49:00.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.167+0000 7f0bd759e640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0bd806ef70 0x7f0bd806f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.167+0000 7f0bd759e640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f0bd81443b0 0x7f0bd8147490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.167+0000 7f0bd759e640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bd80714b0 con 0x7f0bd8143f20 2024-04-25T23:49:00.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.167+0000 7f0bd759e640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f0bd80702b0 con 0x7f0bd806ef70 2024-04-25T23:49:00.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.167+0000 7f0bd759e640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f0bd81479d0 con 0x7f0bd81443b0 2024-04-25T23:49:00.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.167+0000 7f0bd5d9b640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f0bd8143f20 0x7f0bd8146d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:49636/0 (socket says 172.21.15.57:49636) 2024-04-25T23:49:00.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.167+0000 7f0bd5d9b640 1 -- 172.21.15.57:0/1788503067 learned_addr learned my addr 172.21.15.57:0/1788503067 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-04-25T23:49:00.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.167+0000 7f0bd6d9d640 1 --2- 172.21.15.57:0/1788503067 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f0bd81443b0 0x7f0bd8147490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.167+0000 7f0bd6d9d640 1 -- 172.21.15.57:0/1788503067 >> v2:172.21.15.80:3300/0 conn(0x7f0bd806ef70 msgr2=0x7f0bd806f350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T23:49:00.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.167+0000 7f0bd6d9d640 1 --2- 172.21.15.57:0/1788503067 >> v2:172.21.15.80:3300/0 conn(0x7f0bd806ef70 0x7f0bd806f350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.167+0000 7f0bd6d9d640 1 -- 172.21.15.57:0/1788503067 >> v1:172.21.15.57:6789/0 conn(0x7f0bd8143f20 legacy=0x7f0bd8146d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.167+0000 7f0bd6d9d640 1 -- 172.21.15.57:0/1788503067 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bd806ff40 con 0x7f0bd81443b0 2024-04-25T23:49:00.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.171+0000 7f2acd379640 1 --2- 172.21.15.57:0/795898583 >> v2:172.21.15.57:6812/2528545516 conn(0x7f2ac813f820 0x7f2ac81e4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.171+0000 7f2acd379640 1 -- 172.21.15.57:0/795898583 --> v2:172.21.15.57:6812/2528545516 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2ac804f510 con 0x7f2ac813f820 2024-04-25T23:49:00.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.171+0000 7f1ecdd83640 1 --2- 172.21.15.57:0/713409486 >> v2:172.21.15.80:6816/2493457677 conn(0x7f1eac0678f0 0x7f1eac069db0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1ec80b7ce0 tx=0x7f1ec000d040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:00.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.171+0000 7fa5ec98e640 1 --2- 172.21.15.57:0/3726924551 >> v2:172.21.15.80:6816/2493457677 conn(0x7fa5cc0678c0 0x7fa5cc069d80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc0b1000 tx=0x7fa5dc004740 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:00.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.171+0000 7fa14ffff640 1 --2- 172.21.15.57:0/1105575273 >> v2:172.21.15.80:6816/2493457677 conn(0x7fa134067820 0x7fa134069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.171+0000 7fa15523c640 1 --2- 172.21.15.57:0/1105575273 >> v2:172.21.15.80:6811/2954555099 conn(0x7fa13406d800 0x7fa13406fc20 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:00.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.171+0000 7fa5e7fff640 1 --2- 172.21.15.57:0/3726924551 >> v2:172.21.15.57:6804/1976583347 conn(0x7fa5cc06d8a0 0x7fa5cc06fcc0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:00.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.171+0000 7f2accb78640 1 --2- 172.21.15.57:0/795898583 >> v2:172.21.15.57:6812/2528545516 conn(0x7f2ac813f820 0x7f2ac81e4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.171+0000 7f2ac7fff640 1 --2- 172.21.15.57:0/795898583 >> v2:172.21.15.80:6816/2493457677 conn(0x7f2ab40678c0 0x7f2ab4069d80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8006fd0 tx=0x7f2ab8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:00.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.171+0000 7fa14ffff640 1 --2- 172.21.15.57:0/1105575273 >> v2:172.21.15.80:6816/2493457677 conn(0x7fa134067820 0x7fa134069ce0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa1440b2000 tx=0x7fa14400a040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:00.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.171+0000 7fc5577fe640 1 --2- 172.21.15.57:0/18981222 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fc5500cc4a0 0x7fc5500ce890 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc558074430 tx=0x7fc558099db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=177db1b11c4349e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:00.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.171+0000 7f2accb78640 1 --2- 172.21.15.57:0/795898583 >> v2:172.21.15.57:6812/2528545516 conn(0x7f2ac813f820 0x7f2ac81e4340 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:00.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.171+0000 7fa5e57fa640 1 -- 172.21.15.57:0/3726924551 <== osd.2 v2:172.21.15.57:6804/1976583347 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa5e8084e50 con 0x7fa5cc06d8a0 2024-04-25T23:49:00.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.171+0000 7fd6e659c640 1 --2- 172.21.15.57:0/2247819301 >> v2:172.21.15.80:6816/2493457677 conn(0x7fd6d0067940 0x7fd6d0069e00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc006fd0 tx=0x7fd6dc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:00.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.171+0000 7f0bd6d9d640 1 --2- 172.21.15.57:0/1788503067 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f0bd81443b0 0x7f0bd8147490 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8009a20 tx=0x7f0bc801c590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c637c0b1d3d2331 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:00.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.171+0000 7fc555ffb640 1 -- 172.21.15.57:0/18981222 <== mon.2 v2:172.21.15.57:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc55812f070 con 0x7fc5500cc4a0 2024-04-25T23:49:00.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.171+0000 7fa14d7fa640 1 -- 172.21.15.57:0/1105575273 <== osd.5 v2:172.21.15.80:6811/2954555099 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa134070290 con 0x7fa13406d800 2024-04-25T23:49:00.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.171+0000 7fc555ffb640 1 -- 172.21.15.57:0/18981222 <== mon.2 v2:172.21.15.57:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc558122c90 con 0x7fc5500cc4a0 2024-04-25T23:49:00.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.171+0000 7fc555ffb640 1 -- 172.21.15.57:0/18981222 <== mon.2 v2:172.21.15.57:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc55812a640 con 0x7fc5500cc4a0 2024-04-25T23:49:00.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.171+0000 7fc55d9ea640 1 -- 172.21.15.57:0/18981222 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fc5500cc4a0 msgr2=0x7fc5500ce890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.171+0000 7fc55d9ea640 1 --2- 172.21.15.57:0/18981222 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fc5500cc4a0 0x7fc5500ce890 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc558074430 tx=0x7fc558099db0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.175+0000 7f2ac4ff9640 1 -- 172.21.15.57:0/795898583 <== osd.0 v2:172.21.15.57:6812/2528545516 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2ac804f510 con 0x7f2ac813f820 2024-04-25T23:49:00.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.171+0000 7f0bd559a640 1 -- 172.21.15.57:0/1788503067 <== mon.2 v2:172.21.15.57:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc80aa650 con 0x7f0bd81443b0 2024-04-25T23:49:00.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.171+0000 7f0bd559a640 1 -- 172.21.15.57:0/1788503067 <== mon.2 v2:172.21.15.57:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bc80af070 con 0x7f0bd81443b0 2024-04-25T23:49:00.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.175+0000 7fc55d9ea640 1 -- 172.21.15.57:0/18981222 shutdown_connections 2024-04-25T23:49:00.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.175+0000 7fc55d9ea640 1 --2- 172.21.15.57:0/18981222 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fc5500cc4a0 0x7fc5500ce890 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.175+0000 7fc55d9ea640 1 --2- 172.21.15.57:0/18981222 >> v2:172.21.15.80:3300/0 conn(0x7fc5500c0ea0 0x7fc5500c12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.175+0000 7fc55d9ea640 1 -- 172.21.15.57:0/18981222 >> 172.21.15.57:0/18981222 conn(0x7fc550005f20 msgr2=0x7fc550006300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:49:00.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.175+0000 7f0bd759e640 1 -- 172.21.15.57:0/1788503067 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f0bd81443b0 msgr2=0x7f0bd8147490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.175+0000 7f0bd759e640 1 --2- 172.21.15.57:0/1788503067 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f0bd81443b0 0x7f0bd8147490 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8009a20 tx=0x7f0bc801c590 comp rx=0 tx=0).stop 2024-04-25T23:49:00.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.175+0000 7f0bd759e640 1 -- 172.21.15.57:0/1788503067 shutdown_connections 2024-04-25T23:49:00.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.175+0000 7f0bd759e640 1 --2- 172.21.15.57:0/1788503067 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f0bd81443b0 0x7f0bd8147490 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.175+0000 7f0bd759e640 1 --2- 172.21.15.57:0/1788503067 >> v2:172.21.15.80:3300/0 conn(0x7f0bd806ef70 0x7f0bd806f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.175+0000 7fc55d9ea640 1 -- 172.21.15.57:0/18981222 shutdown_connections 2024-04-25T23:49:00.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.175+0000 7f0bd759e640 1 -- 172.21.15.57:0/1788503067 >> 172.21.15.57:0/1788503067 conn(0x7f0bd80b42a0 msgr2=0x7f0bd80b46b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:49:00.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.175+0000 7fc55d9ea640 1 -- 172.21.15.57:0/18981222 wait complete. 2024-04-25T23:49:00.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.175+0000 7fc55d9ea640 1 Processor -- start 2024-04-25T23:49:00.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.175+0000 7fc55d9ea640 1 -- start start 2024-04-25T23:49:00.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.175+0000 7fc55d9ea640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc550034cb0 0x7fc55002d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.175+0000 7fc55d9ea640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fc5500c0ea0 0x7fc55002d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.175+0000 7fc55d9ea640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5500cc4a0 0x7fc550164bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.175+0000 7fc55d9ea640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fc550031b70 con 0x7fc5500cc4a0 2024-04-25T23:49:00.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.175+0000 7fc55d9ea640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc5500319f0 con 0x7fc550034cb0 2024-04-25T23:49:00.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.175+0000 7fc55d9ea640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7fc5500c42f0 con 0x7fc5500c0ea0 2024-04-25T23:49:00.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.179+0000 7fc5567fc640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fc5500c0ea0 0x7fc55002d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.179+0000 7fc5567fc640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fc5500c0ea0 0x7fc55002d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3301/0 says I am v2:172.21.15.57:47360/0 (socket says 172.21.15.57:47360) 2024-04-25T23:49:00.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.179+0000 7fc5567fc640 1 -- 172.21.15.57:0/23716104 learned_addr learned my addr 172.21.15.57:0/23716104 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:49:00.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.179+0000 7fc556ffd640 1 --2- 172.21.15.57:0/23716104 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc550034cb0 0x7fc55002d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.175+0000 7f0bd759e640 1 -- 172.21.15.57:0/1788503067 shutdown_connections 2024-04-25T23:49:00.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.179+0000 7fc5577fe640 1 --2- 172.21.15.57:0/23716104 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5500cc4a0 0x7fc550164bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.211 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.175+0000 7f0bd759e640 1 -- 172.21.15.57:0/1788503067 wait complete. 2024-04-25T23:49:00.211 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.179+0000 7fc5567fc640 1 -- 172.21.15.57:0/23716104 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc550034cb0 msgr2=0x7fc55002d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.211 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.179+0000 7fc5567fc640 1 --2- 172.21.15.57:0/23716104 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc550034cb0 0x7fc55002d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.211 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.179+0000 7fc5567fc640 1 -- 172.21.15.57:0/23716104 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5500cc4a0 msgr2=0x7fc550164bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.211 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.179+0000 7fc5567fc640 1 --2- 172.21.15.57:0/23716104 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5500cc4a0 0x7fc550164bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.179+0000 7fc5567fc640 1 -- 172.21.15.57:0/23716104 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc55002e940 con 0x7fc5500c0ea0 2024-04-25T23:49:00.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.179+0000 7fc5577fe640 1 --2- 172.21.15.57:0/23716104 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5500cc4a0 0x7fc550164bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T23:49:00.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.175+0000 7f0bd759e640 1 Processor -- start 2024-04-25T23:49:00.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.175+0000 7f0bd759e640 1 -- start start 2024-04-25T23:49:00.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.175+0000 7f0bd759e640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f0bd806ef70 0x7f0bd8144960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.175+0000 7f0bd759e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0bd8143f20 0x7f0bd813db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.213 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.179+0000 7fc5567fc640 1 --2- 172.21.15.57:0/23716104 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fc5500c0ea0 0x7fc55002d590 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc54c00b4d0 tx=0x7fc54c00b990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:00.213 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.175+0000 7f0bd759e640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0bd81443b0 0x7f0bd80b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.213 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.175+0000 7f0bd759e640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd81515f0 con 0x7f0bd81443b0 2024-04-25T23:49:00.213 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.175+0000 7f0bd759e640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd8151470 con 0x7f0bd8143f20 2024-04-25T23:49:00.214 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.179+0000 7fc53ffff640 1 -- 172.21.15.57:0/23716104 <== mon.2 v2:172.21.15.57:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc54c014070 con 0x7fc5500c0ea0 2024-04-25T23:49:00.214 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.175+0000 7f0bd759e640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f0bd8151770 con 0x7f0bd806ef70 2024-04-25T23:49:00.214 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.175+0000 7f0bd5d9b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0bd8143f20 0x7f0bd813db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.214 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.175+0000 7f0bd5d9b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0bd8143f20 0x7f0bd813db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.57:54252/0 (socket says 172.21.15.57:54252) 2024-04-25T23:49:00.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.179+0000 7fc55d9ea640 1 -- 172.21.15.57:0/23716104 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5500299f0 con 0x7fc5500c0ea0 2024-04-25T23:49:00.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.179+0000 7fc55d9ea640 1 -- 172.21.15.57:0/23716104 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc55002c4b0 con 0x7fc5500c0ea0 2024-04-25T23:49:00.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.179+0000 7fc53ffff640 1 -- 172.21.15.57:0/23716104 <== mon.2 v2:172.21.15.57:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc54c010890 con 0x7fc5500c0ea0 2024-04-25T23:49:00.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.179+0000 7fc53ffff640 1 -- 172.21.15.57:0/23716104 <== mon.2 v2:172.21.15.57:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc54c010c80 con 0x7fc5500c0ea0 2024-04-25T23:49:00.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.175+0000 7f0bd5d9b640 1 -- 172.21.15.57:0/782695326 learned_addr learned my addr 172.21.15.57:0/782695326 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:49:00.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.175+0000 7f0bd6d9d640 1 --2- 172.21.15.57:0/782695326 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0bd81443b0 0x7f0bd80b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.175+0000 7f0bd5d9b640 1 -- 172.21.15.57:0/782695326 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f0bd806ef70 msgr2=0x7f0bd8144960 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T23:49:00.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.175+0000 7f0bd5d9b640 1 --2- 172.21.15.57:0/782695326 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f0bd806ef70 0x7f0bd8144960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.175+0000 7f0bd5d9b640 1 -- 172.21.15.57:0/782695326 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0bd81443b0 msgr2=0x7f0bd80b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.175+0000 7f0bd5d9b640 1 --2- 172.21.15.57:0/782695326 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0bd81443b0 0x7f0bd80b2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.175+0000 7f0bd5d9b640 1 -- 172.21.15.57:0/782695326 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bd80b1e10 con 0x7f0bd8143f20 2024-04-25T23:49:00.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.175+0000 7f0bd6d9d640 1 --2- 172.21.15.57:0/782695326 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0bd81443b0 0x7f0bd80b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T23:49:00.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.179+0000 7f0bd5d9b640 1 --2- 172.21.15.57:0/782695326 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0bd8143f20 0x7f0bd813db80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0bd000e7b0 tx=0x7f0bd000ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:00.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.179+0000 7f0bc6ffd640 1 -- 172.21.15.57:0/782695326 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd000f930 con 0x7f0bd8143f20 2024-04-25T23:49:00.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.179+0000 7f0bc6ffd640 1 -- 172.21.15.57:0/782695326 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bd0013070 con 0x7f0bd8143f20 2024-04-25T23:49:00.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.179+0000 7f0bc6ffd640 1 -- 172.21.15.57:0/782695326 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd000fcd0 con 0x7f0bd8143f20 2024-04-25T23:49:00.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.179+0000 7f0bd759e640 1 -- 172.21.15.57:0/782695326 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bd811ba80 con 0x7f0bd8143f20 2024-04-25T23:49:00.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.179+0000 7f0bd759e640 1 -- 172.21.15.57:0/782695326 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bd813e0c0 con 0x7f0bd8143f20 2024-04-25T23:49:00.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.179+0000 7f0bc6ffd640 1 -- 172.21.15.57:0/782695326 <== mon.1 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 4) ==== 87354+0+0 (secure 0 0 0) 0x7f0bd0018470 con 0x7f0bd8143f20 2024-04-25T23:49:00.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.183+0000 7f0bc6ffd640 1 --2- 172.21.15.57:0/782695326 >> v2:172.21.15.80:6816/2493457677 conn(0x7f0bc0067a10 0x7f0bc0069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.183+0000 7f0bc6ffd640 1 -- 172.21.15.57:0/782695326 <== mon.1 v2:172.21.15.80:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4090+0+0 (secure 0 0 0) 0x7f0bd0087f90 con 0x7f0bd8143f20 2024-04-25T23:49:00.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.183+0000 7f0bd659c640 1 --2- 172.21.15.57:0/782695326 >> v2:172.21.15.80:6816/2493457677 conn(0x7f0bc0067a10 0x7f0bc0069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.219 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.183+0000 7fc53ffff640 1 -- 172.21.15.57:0/23716104 <== mon.2 v2:172.21.15.57:3301/0 4 ==== mgrmap(e 4) ==== 87354+0+0 (secure 0 0 0) 0x7fc54c017070 con 0x7fc5500c0ea0 2024-04-25T23:49:00.219 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.183+0000 7fc53ffff640 1 --2- 172.21.15.57:0/23716104 >> v2:172.21.15.80:6816/2493457677 conn(0x7fc540067a10 0x7fc540069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.219 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.183+0000 7fc53ffff640 1 -- 172.21.15.57:0/23716104 <== mon.2 v2:172.21.15.57:3301/0 5 ==== osd_map(17..17 src has 1..17) ==== 4090+0+0 (secure 0 0 0) 0x7fc54c085220 con 0x7fc5500c0ea0 2024-04-25T23:49:00.219 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.183+0000 7fc556ffd640 1 --2- 172.21.15.57:0/23716104 >> v2:172.21.15.80:6816/2493457677 conn(0x7fc540067a10 0x7fc540069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.219 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.183+0000 7f0bd759e640 1 --2- 172.21.15.57:0/782695326 >> v2:172.21.15.80:6801/163896711 conn(0x7f0bbc001640 0x7f0bbc003b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.219 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.183+0000 7f0bd759e640 1 -- 172.21.15.57:0/782695326 --> v2:172.21.15.80:6801/163896711 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0bbc006ab0 con 0x7f0bbc001640 2024-04-25T23:49:00.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.183+0000 7fc55d9ea640 1 --2- 172.21.15.57:0/23716104 >> v2:172.21.15.80:6808/2647667231 conn(0x7fc544001640 0x7fc544003b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.183+0000 7fc55d9ea640 1 -- 172.21.15.57:0/23716104 --> v2:172.21.15.80:6808/2647667231 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc544006ab0 con 0x7fc544001640 2024-04-25T23:49:00.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.183+0000 7f0bd6d9d640 1 --2- 172.21.15.57:0/782695326 >> v2:172.21.15.80:6801/163896711 conn(0x7f0bbc001640 0x7f0bbc003b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.183+0000 7fc556ffd640 1 --2- 172.21.15.57:0/23716104 >> v2:172.21.15.80:6816/2493457677 conn(0x7fc540067a10 0x7fc540069ed0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc548007fd0 tx=0x7fc548009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:00.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.183+0000 7fc5577fe640 1 --2- 172.21.15.57:0/23716104 >> v2:172.21.15.80:6808/2647667231 conn(0x7fc544001640 0x7fc544003b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.183+0000 7f0bd6d9d640 1 --2- 172.21.15.57:0/782695326 >> v2:172.21.15.80:6801/163896711 conn(0x7f0bbc001640 0x7f0bbc003b00 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:00.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.183+0000 7fc5577fe640 1 --2- 172.21.15.57:0/23716104 >> v2:172.21.15.80:6808/2647667231 conn(0x7fc544001640 0x7fc544003b00 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:00.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.187+0000 7f0bd659c640 1 --2- 172.21.15.57:0/782695326 >> v2:172.21.15.80:6816/2493457677 conn(0x7f0bc0067a10 0x7f0bc0069ed0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc009690 tx=0x7f0bcc006be0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:00.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.187+0000 7fd6e759e640 1 -- 172.21.15.57:0/2247819301 --> v2:172.21.15.80:6800/4069821322 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd6cc002c00 con 0x7fd6d006d920 2024-04-25T23:49:00.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.187+0000 7f0bc6ffd640 1 -- 172.21.15.57:0/782695326 <== osd.6 v2:172.21.15.80:6801/163896711 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0bbc006ab0 con 0x7f0bbc001640 2024-04-25T23:49:00.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.187+0000 7fd6d6ffd640 1 -- 172.21.15.57:0/2247819301 <== osd.4 v2:172.21.15.80:6800/4069821322 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd6cc002c00 con 0x7fd6d006d920 2024-04-25T23:49:00.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.191+0000 7fd6e759e640 1 -- 172.21.15.57:0/2247819301 >> v2:172.21.15.80:6800/4069821322 conn(0x7fd6d006d920 msgr2=0x7fd6d006fd40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.191+0000 7fd6e759e640 1 --2- 172.21.15.57:0/2247819301 >> v2:172.21.15.80:6800/4069821322 conn(0x7fd6d006d920 0x7fd6d006fd40 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.191+0000 7fd6e759e640 1 -- 172.21.15.57:0/2247819301 >> v2:172.21.15.80:6816/2493457677 conn(0x7fd6d0067940 msgr2=0x7fd6d0069e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.191+0000 7fd6e759e640 1 --2- 172.21.15.57:0/2247819301 >> v2:172.21.15.80:6816/2493457677 conn(0x7fd6d0067940 0x7fd6d0069e00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc006fd0 tx=0x7fd6dc008040 comp rx=0 tx=0).stop 2024-04-25T23:49:00.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.191+0000 7fd6e759e640 1 -- 172.21.15.57:0/2247819301 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd6e8142490 msgr2=0x7fd6e80b4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.191+0000 7fd6e759e640 1 --2- 172.21.15.57:0/2247819301 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd6e8142490 0x7fd6e80b4330 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd6e001a4f0 tx=0x7fd6e0019660 comp rx=0 tx=0).stop 2024-04-25T23:49:00.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.191+0000 7fd6e6d9d640 1 -- 172.21.15.57:0/2247819301 reap_dead start 2024-04-25T23:49:00.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.191+0000 7fd6e759e640 1 -- 172.21.15.57:0/2247819301 shutdown_connections 2024-04-25T23:49:00.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.191+0000 7fd6e759e640 1 -- 172.21.15.57:0/2247819301 >> 172.21.15.57:0/2247819301 conn(0x7fd6e80b4dd0 msgr2=0x7fd6e80adfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:49:00.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.191+0000 7fd6e759e640 1 -- 172.21.15.57:0/2247819301 shutdown_connections 2024-04-25T23:49:00.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.191+0000 7fd6e759e640 1 -- 172.21.15.57:0/2247819301 wait complete. 2024-04-25T23:49:00.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.195+0000 7f1ed07e8640 1 -- 172.21.15.57:0/713409486 --> v2:172.21.15.57:6800/1899803014 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1eb0002c80 con 0x7f1eac06d8d0 2024-04-25T23:49:00.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.195+0000 7f1ebeffd640 1 -- 172.21.15.57:0/713409486 <== osd.3 v2:172.21.15.57:6800/1899803014 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1eb0002c80 con 0x7f1eac06d8d0 2024-04-25T23:49:00.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.195+0000 7f1ed07e8640 1 -- 172.21.15.57:0/713409486 >> v2:172.21.15.57:6800/1899803014 conn(0x7f1eac06d8d0 msgr2=0x7f1eac06fcf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.195+0000 7f1ed07e8640 1 --2- 172.21.15.57:0/713409486 >> v2:172.21.15.57:6800/1899803014 conn(0x7f1eac06d8d0 0x7f1eac06fcf0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.195+0000 7f1ed07e8640 1 -- 172.21.15.57:0/713409486 >> v2:172.21.15.80:6816/2493457677 conn(0x7f1eac0678f0 msgr2=0x7f1eac069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.195+0000 7f1ed07e8640 1 --2- 172.21.15.57:0/713409486 >> v2:172.21.15.80:6816/2493457677 conn(0x7f1eac0678f0 0x7f1eac069db0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1ec80b7ce0 tx=0x7f1ec000d040 comp rx=0 tx=0).stop 2024-04-25T23:49:00.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.195+0000 7f1ed07e8640 1 -- 172.21.15.57:0/713409486 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1ec80a9e50 msgr2=0x7f1ec80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.195+0000 7f1ed07e8640 1 --2- 172.21.15.57:0/713409486 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1ec80a9e50 0x7f1ec80b77a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1ec400b550 tx=0x7f1ec400ba10 comp rx=0 tx=0).stop 2024-04-25T23:49:00.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.195+0000 7f1eced85640 1 -- 172.21.15.57:0/713409486 reap_dead start 2024-04-25T23:49:00.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.195+0000 7f1ed07e8640 1 -- 172.21.15.57:0/713409486 shutdown_connections 2024-04-25T23:49:00.228 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.195+0000 7f1ed07e8640 1 -- 172.21.15.57:0/713409486 >> 172.21.15.57:0/713409486 conn(0x7f1ec8074af0 msgr2=0x7f1ec8137a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:49:00.228 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.195+0000 7f1ed07e8640 1 -- 172.21.15.57:0/713409486 shutdown_connections 2024-04-25T23:49:00.228 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.195+0000 7f1ed07e8640 1 -- 172.21.15.57:0/713409486 wait complete. 2024-04-25T23:49:00.228 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.199+0000 7fbc0a926640 1 -- 172.21.15.57:0/2020037778 --> v2:172.21.15.57:6808/2225240106 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbbec005b80 con 0x7fbbec001640 2024-04-25T23:49:00.228 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.199+0000 7fbbfa7fc640 1 -- 172.21.15.57:0/2020037778 <== osd.1 v2:172.21.15.57:6808/2225240106 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fbbec005b80 con 0x7fbbec001640 2024-04-25T23:49:00.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.199+0000 7fbc0a926640 1 -- 172.21.15.57:0/2020037778 >> v2:172.21.15.57:6808/2225240106 conn(0x7fbbec001640 msgr2=0x7fbbec003b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.199+0000 7fbc0a926640 1 --2- 172.21.15.57:0/2020037778 >> v2:172.21.15.57:6808/2225240106 conn(0x7fbbec001640 0x7fbbec003b00 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.203+0000 7fbc0a926640 1 -- 172.21.15.57:0/2020037778 >> v2:172.21.15.80:6816/2493457677 conn(0x7fbbe80678c0 msgr2=0x7fbbe8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.203+0000 7fbc0a926640 1 --2- 172.21.15.57:0/2020037778 >> v2:172.21.15.80:6816/2493457677 conn(0x7fbbe80678c0 0x7fbbe8069d80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbc00007fd0 tx=0x7fbc00007d80 comp rx=0 tx=0).stop 2024-04-25T23:49:00.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.203+0000 7fbc0a926640 1 -- 172.21.15.57:0/2020037778 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbbfc036e30 msgr2=0x7fbbfc0c5ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.203+0000 7fbc0a926640 1 --2- 172.21.15.57:0/2020037778 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbbfc036e30 0x7fbbfc0c5ee0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbc040689b0 tx=0x7fbc040685e0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.203+0000 7fbc0a125640 1 -- 172.21.15.57:0/2020037778 reap_dead start 2024-04-25T23:49:00.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.203+0000 7fbc0a926640 1 -- 172.21.15.57:0/2020037778 shutdown_connections 2024-04-25T23:49:00.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.203+0000 7fbc0a926640 1 -- 172.21.15.57:0/2020037778 >> 172.21.15.57:0/2020037778 conn(0x7fbbfc0063f0 msgr2=0x7fbbfc009700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:49:00.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.203+0000 7fbc0a926640 1 -- 172.21.15.57:0/2020037778 shutdown_connections 2024-04-25T23:49:00.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.203+0000 7fbc0a926640 1 -- 172.21.15.57:0/2020037778 wait complete. 2024-04-25T23:49:00.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.203+0000 7fa5ed990640 1 -- 172.21.15.57:0/3726924551 --> v2:172.21.15.57:6804/1976583347 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa5c8002c80 con 0x7fa5cc06d8a0 2024-04-25T23:49:00.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.203+0000 7fa5e57fa640 1 -- 172.21.15.57:0/3726924551 <== osd.2 v2:172.21.15.57:6804/1976583347 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa5c8002c80 con 0x7fa5cc06d8a0 2024-04-25T23:49:00.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.203+0000 7fa5ed990640 1 -- 172.21.15.57:0/3726924551 >> v2:172.21.15.57:6804/1976583347 conn(0x7fa5cc06d8a0 msgr2=0x7fa5cc06fcc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.203+0000 7fa5ed990640 1 --2- 172.21.15.57:0/3726924551 >> v2:172.21.15.57:6804/1976583347 conn(0x7fa5cc06d8a0 0x7fa5cc06fcc0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.203+0000 7fa5ed990640 1 -- 172.21.15.57:0/3726924551 >> v2:172.21.15.80:6816/2493457677 conn(0x7fa5cc0678c0 msgr2=0x7fa5cc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.203+0000 7fa5ed990640 1 --2- 172.21.15.57:0/3726924551 >> v2:172.21.15.80:6816/2493457677 conn(0x7fa5cc0678c0 0x7fa5cc069d80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc0b1000 tx=0x7fa5dc004740 comp rx=0 tx=0).stop 2024-04-25T23:49:00.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.203+0000 7fa5ed990640 1 -- 172.21.15.57:0/3726924551 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5e0037610 msgr2=0x7fa5e0163470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.203+0000 7fa5ed990640 1 --2- 172.21.15.57:0/3726924551 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5e0037610 0x7fa5e0163470 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa5e802d710 tx=0x7fa5e80685e0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.203+0000 7fa5ed18f640 1 -- 172.21.15.57:0/3726924551 reap_dead start 2024-04-25T23:49:00.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.203+0000 7fa5ed990640 1 -- 172.21.15.57:0/3726924551 shutdown_connections 2024-04-25T23:49:00.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.203+0000 7fa5ed990640 1 -- 172.21.15.57:0/3726924551 >> 172.21.15.57:0/3726924551 conn(0x7fa5e003a8d0 msgr2=0x7fa5e00c0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:49:00.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.203+0000 7fa5ed990640 1 -- 172.21.15.57:0/3726924551 shutdown_connections 2024-04-25T23:49:00.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.203+0000 7fa156c9f640 1 -- 172.21.15.57:0/1105575273 --> v2:172.21.15.80:6811/2954555099 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa150139430 con 0x7fa13406d800 2024-04-25T23:49:00.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.203+0000 7fa5ed990640 1 -- 172.21.15.57:0/3726924551 wait complete. 2024-04-25T23:49:00.234 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.203+0000 7fa14d7fa640 1 -- 172.21.15.57:0/1105575273 <== osd.5 v2:172.21.15.80:6811/2954555099 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa150139430 con 0x7fa13406d800 2024-04-25T23:49:00.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.207+0000 7fa156c9f640 1 -- 172.21.15.57:0/1105575273 >> v2:172.21.15.80:6811/2954555099 conn(0x7fa13406d800 msgr2=0x7fa13406fc20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.207+0000 7fa156c9f640 1 --2- 172.21.15.57:0/1105575273 >> v2:172.21.15.80:6811/2954555099 conn(0x7fa13406d800 0x7fa13406fc20 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.207+0000 7fa156c9f640 1 -- 172.21.15.57:0/1105575273 >> v2:172.21.15.80:6816/2493457677 conn(0x7fa134067820 msgr2=0x7fa134069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.207+0000 7fa156c9f640 1 --2- 172.21.15.57:0/1105575273 >> v2:172.21.15.80:6816/2493457677 conn(0x7fa134067820 0x7fa134069ce0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa1440b2000 tx=0x7fa14400a040 comp rx=0 tx=0).stop 2024-04-25T23:49:00.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.207+0000 7fa156c9f640 1 -- 172.21.15.57:0/1105575273 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fa150070140 msgr2=0x7fa15013cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.207+0000 7fa156c9f640 1 --2- 172.21.15.57:0/1105575273 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fa150070140 0x7fa15013cdb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa13c00b4a0 tx=0x7fa13c00b960 comp rx=0 tx=0).stop 2024-04-25T23:49:00.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.207+0000 7fa15523c640 1 -- 172.21.15.57:0/1105575273 reap_dead start 2024-04-25T23:49:00.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.207+0000 7fa156c9f640 1 -- 172.21.15.57:0/1105575273 shutdown_connections 2024-04-25T23:49:00.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.207+0000 7fa156c9f640 1 -- 172.21.15.57:0/1105575273 >> 172.21.15.57:0/1105575273 conn(0x7fa1500b4480 msgr2=0x7fa1500af460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:49:00.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.207+0000 7fa156c9f640 1 -- 172.21.15.57:0/1105575273 shutdown_connections 2024-04-25T23:49:00.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.207+0000 7fa156c9f640 1 -- 172.21.15.57:0/1105575273 wait complete. 2024-04-25T23:49:00.236 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.215+0000 7f2acd379640 1 -- 172.21.15.57:0/795898583 --> v2:172.21.15.57:6812/2528545516 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2ac80af650 con 0x7f2ac813f820 2024-04-25T23:49:00.237 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.215+0000 7f2ac4ff9640 1 -- 172.21.15.57:0/795898583 <== osd.0 v2:172.21.15.57:6812/2528545516 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2ac80af650 con 0x7f2ac813f820 2024-04-25T23:49:00.237 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.215+0000 7f2acd379640 1 -- 172.21.15.57:0/795898583 >> v2:172.21.15.57:6812/2528545516 conn(0x7f2ac813f820 msgr2=0x7f2ac81e4340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.215+0000 7f2acd379640 1 --2- 172.21.15.57:0/795898583 >> v2:172.21.15.57:6812/2528545516 conn(0x7f2ac813f820 0x7f2ac81e4340 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.215+0000 7f2acd379640 1 -- 172.21.15.57:0/795898583 >> v2:172.21.15.80:6816/2493457677 conn(0x7f2ab40678c0 msgr2=0x7f2ab4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.215+0000 7f2acd379640 1 --2- 172.21.15.57:0/795898583 >> v2:172.21.15.80:6816/2493457677 conn(0x7f2ab40678c0 0x7f2ab4069d80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8006fd0 tx=0x7f2ab8008040 comp rx=0 tx=0).stop 2024-04-25T23:49:00.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.215+0000 7f2acd379640 1 -- 172.21.15.57:0/795898583 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ac806f1e0 msgr2=0x7f2ac80b1c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.215+0000 7f2acd379640 1 --2- 172.21.15.57:0/795898583 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ac806f1e0 0x7f2ac80b1c30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2abc0ae000 tx=0x7f2abc01b410 comp rx=0 tx=0).stop 2024-04-25T23:49:00.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.215+0000 7f2accb78640 1 -- 172.21.15.57:0/795898583 reap_dead start 2024-04-25T23:49:00.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.215+0000 7f2acd379640 1 -- 172.21.15.57:0/795898583 shutdown_connections 2024-04-25T23:49:00.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.215+0000 7f2acd379640 1 -- 172.21.15.57:0/795898583 >> 172.21.15.57:0/795898583 conn(0x7f2ac80752a0 msgr2=0x7f2ac80aee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:49:00.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.215+0000 7f2acd379640 1 -- 172.21.15.57:0/795898583 shutdown_connections 2024-04-25T23:49:00.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.215+0000 7f2acd379640 1 -- 172.21.15.57:0/795898583 wait complete. 2024-04-25T23:49:00.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.219+0000 7f0bd759e640 1 -- 172.21.15.57:0/782695326 --> v2:172.21.15.80:6801/163896711 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0bbc005b80 con 0x7f0bbc001640 2024-04-25T23:49:00.239 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.219+0000 7f0bc6ffd640 1 -- 172.21.15.57:0/782695326 <== osd.6 v2:172.21.15.80:6801/163896711 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0bbc005b80 con 0x7f0bbc001640 2024-04-25T23:49:00.240 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.219+0000 7f0bd759e640 1 -- 172.21.15.57:0/782695326 >> v2:172.21.15.80:6801/163896711 conn(0x7f0bbc001640 msgr2=0x7f0bbc003b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.240 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.219+0000 7f0bd759e640 1 --2- 172.21.15.57:0/782695326 >> v2:172.21.15.80:6801/163896711 conn(0x7f0bbc001640 0x7f0bbc003b00 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.240 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.219+0000 7f0bd759e640 1 -- 172.21.15.57:0/782695326 >> v2:172.21.15.80:6816/2493457677 conn(0x7f0bc0067a10 msgr2=0x7f0bc0069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.240 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.219+0000 7f0bd759e640 1 --2- 172.21.15.57:0/782695326 >> v2:172.21.15.80:6816/2493457677 conn(0x7f0bc0067a10 0x7f0bc0069ed0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc009690 tx=0x7f0bcc006be0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.240 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.219+0000 7f0bd759e640 1 -- 172.21.15.57:0/782695326 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0bd8143f20 msgr2=0x7f0bd813db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.240 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.219+0000 7f0bd759e640 1 --2- 172.21.15.57:0/782695326 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0bd8143f20 0x7f0bd813db80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0bd000e7b0 tx=0x7f0bd000ec70 comp rx=0 tx=0).stop 2024-04-25T23:49:00.241 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.219+0000 7f0bd6d9d640 1 -- 172.21.15.57:0/782695326 reap_dead start 2024-04-25T23:49:00.241 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.219+0000 7f0bd759e640 1 -- 172.21.15.57:0/782695326 shutdown_connections 2024-04-25T23:49:00.241 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.219+0000 7f0bd759e640 1 -- 172.21.15.57:0/782695326 >> 172.21.15.57:0/782695326 conn(0x7f0bd80b42a0 msgr2=0x7f0bd8146c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:49:00.241 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.219+0000 7f0bd759e640 1 -- 172.21.15.57:0/782695326 shutdown_connections 2024-04-25T23:49:00.241 INFO:teuthology.orchestra.run.smithi057.stdout:55834574850 2024-04-25T23:49:00.241 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.219+0000 7f0bd759e640 1 -- 172.21.15.57:0/782695326 wait complete. 2024-04-25T23:49:00.242 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T23:49:00.243 INFO:teuthology.orchestra.run.smithi057.stdout:60129542146 2024-04-25T23:49:00.243 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T23:49:00.244 INFO:teuthology.orchestra.run.smithi057.stdout:55834574850 2024-04-25T23:49:00.245 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T23:49:00.246 INFO:teuthology.orchestra.run.smithi057.stdout:55834574850 2024-04-25T23:49:00.246 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T23:49:00.247 INFO:teuthology.orchestra.run.smithi057.stdout:60129542147 2024-04-25T23:49:00.247 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T23:49:00.248 INFO:teuthology.orchestra.run.smithi057.stdout:55834574850 2024-04-25T23:49:00.248 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T23:49:00.249 INFO:teuthology.orchestra.run.smithi057.stdout:60129542146 2024-04-25T23:49:00.249 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T23:49:00.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.283+0000 7fc53ffff640 1 -- 172.21.15.57:0/23716104 <== osd.7 v2:172.21.15.80:6808/2647667231 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc544006ab0 con 0x7fc544001640 2024-04-25T23:49:00.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.319+0000 7fc55d9ea640 1 -- 172.21.15.57:0/23716104 --> v2:172.21.15.80:6808/2647667231 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc544005b80 con 0x7fc544001640 2024-04-25T23:49:00.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.323+0000 7fc53ffff640 1 -- 172.21.15.57:0/23716104 <== osd.7 v2:172.21.15.80:6808/2647667231 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc544005b80 con 0x7fc544001640 2024-04-25T23:49:00.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.339+0000 7fc53dffb640 1 -- 172.21.15.57:0/23716104 >> v2:172.21.15.80:6808/2647667231 conn(0x7fc544001640 msgr2=0x7fc544003b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.339+0000 7fc53dffb640 1 --2- 172.21.15.57:0/23716104 >> v2:172.21.15.80:6808/2647667231 conn(0x7fc544001640 0x7fc544003b00 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.339+0000 7fc53dffb640 1 -- 172.21.15.57:0/23716104 >> v2:172.21.15.80:6816/2493457677 conn(0x7fc540067a10 msgr2=0x7fc540069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.339+0000 7fc53dffb640 1 --2- 172.21.15.57:0/23716104 >> v2:172.21.15.80:6816/2493457677 conn(0x7fc540067a10 0x7fc540069ed0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc548007fd0 tx=0x7fc548009040 comp rx=0 tx=0).stop 2024-04-25T23:49:00.327 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.339+0000 7fc53dffb640 1 -- 172.21.15.57:0/23716104 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fc5500c0ea0 msgr2=0x7fc55002d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.327 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.339+0000 7fc53dffb640 1 --2- 172.21.15.57:0/23716104 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fc5500c0ea0 0x7fc55002d590 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc54c00b4d0 tx=0x7fc54c00b990 comp rx=0 tx=0).stop 2024-04-25T23:49:00.327 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.339+0000 7fc5577fe640 1 -- 172.21.15.57:0/23716104 reap_dead start 2024-04-25T23:49:00.327 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.339+0000 7fc53dffb640 1 -- 172.21.15.57:0/23716104 shutdown_connections 2024-04-25T23:49:00.327 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.339+0000 7fc53dffb640 1 -- 172.21.15.57:0/23716104 >> 172.21.15.57:0/23716104 conn(0x7fc550005f20 msgr2=0x7fc550006300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:49:00.327 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.339+0000 7fc53dffb640 1 -- 172.21.15.57:0/23716104 shutdown_connections 2024-04-25T23:49:00.327 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.339+0000 7fc53dffb640 1 -- 172.21.15.57:0/23716104 wait complete. 2024-04-25T23:49:00.354 INFO:teuthology.orchestra.run.smithi057.stdout:55834574850 2024-04-25T23:49:00.355 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T23:49:00.452 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.463+0000 7f615359e640 1 Processor -- start 2024-04-25T23:49:00.453 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.463+0000 7f615359e640 1 -- start start 2024-04-25T23:49:00.454 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.467+0000 7f615359e640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f615413bcb0 0x7f615413e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.454 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.467+0000 7f615359e640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6154073310 con 0x7f615406f420 2024-04-25T23:49:00.454 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.467+0000 7f615359e640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6154071970 con 0x7f615413ec50 2024-04-25T23:49:00.454 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.467+0000 7f615359e640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f6154071720 con 0x7f615413bcb0 2024-04-25T23:49:00.455 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.467+0000 7f615259c640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f615406f420 0x7f615413b5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:49650/0 (socket says 172.21.15.57:49650) 2024-04-25T23:49:00.455 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.467+0000 7f615259c640 1 -- 172.21.15.57:0/2597764742 learned_addr learned my addr 172.21.15.57:0/2597764742 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-04-25T23:49:00.455 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.467+0000 7f6151d9b640 1 --2- 172.21.15.57:0/2597764742 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f615413bcb0 0x7f615413e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.455 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.467+0000 7f3c1bbda640 1 Processor -- start 2024-04-25T23:49:00.455 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.467+0000 7f3c1bbda640 1 -- start start 2024-04-25T23:49:00.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.467+0000 7f615159a640 1 -- 172.21.15.57:0/2597764742 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2873367164 0 0) 0x7f6154073310 con 0x7f615406f420 2024-04-25T23:49:00.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.467+0000 7f615159a640 1 -- 172.21.15.57:0/2597764742 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f613c003530 con 0x7f615406f420 2024-04-25T23:49:00.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.467+0000 7f3c1bbda640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f3c1406efb0 0x7f3c14143fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.467+0000 7f3c1bbda640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3c1406f390 0x7f3c14151490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.467+0000 7f3c1bbda640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c14071210 con 0x7f3c14144500 2024-04-25T23:49:00.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.467+0000 7f3c1bbda640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f3c14146c10 con 0x7f3c1406f390 2024-04-25T23:49:00.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.467+0000 7f3c1bbda640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f3c1406fb30 con 0x7f3c1406efb0 2024-04-25T23:49:00.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.467+0000 7f3c19976640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f3c1406efb0 0x7f3c14143fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.467+0000 7f615159a640 1 -- 172.21.15.57:0/2597764742 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1645536745 0 0) 0x7f613c003530 con 0x7f615406f420 2024-04-25T23:49:00.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.467+0000 7f615159a640 1 -- 172.21.15.57:0/2597764742 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f615413bcb0 msgr2=0x7f615413e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:49:00.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.467+0000 7f615159a640 1 --2- 172.21.15.57:0/2597764742 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f615413bcb0 0x7f615413e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.467+0000 7f615159a640 1 -- 172.21.15.57:0/2597764742 >> v1:172.21.15.80:6789/0 conn(0x7f615413ec50 legacy=0x7f6154149020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.457 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.467+0000 7f615159a640 1 -- 172.21.15.57:0/2597764742 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f615413c250 con 0x7f615406f420 2024-04-25T23:49:00.458 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.467+0000 7f3c1a177640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3c1406f390 0x7f3c14151490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.458 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.467+0000 7f3c1a177640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3c1406f390 0x7f3c14151490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.57:54272/0 (socket says 172.21.15.57:54272) 2024-04-25T23:49:00.458 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.467+0000 7f3c1a177640 1 -- 172.21.15.57:0/1709096961 learned_addr learned my addr 172.21.15.57:0/1709096961 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:49:00.458 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.467+0000 7f3c19976640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f3c1406efb0 0x7f3c14143fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3301/0 says I am v2:172.21.15.57:47376/0 (socket says 172.21.15.57:47376) 2024-04-25T23:49:00.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.467+0000 7f615159a640 1 -- 172.21.15.57:0/2597764742 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2627952221 0 0) 0x7f6144000ee0 con 0x7f615406f420 2024-04-25T23:49:00.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.467+0000 7f615159a640 1 -- 172.21.15.57:0/2597764742 >> v1:172.21.15.57:6789/0 conn(0x7f615406f420 legacy=0x7f615413b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.467+0000 7f615159a640 1 --2- 172.21.15.57:0/2597764742 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f613c003ac0 0x7f613c003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.467+0000 7f615159a640 1 --2- 172.21.15.57:0/2597764742 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f613c004810 0x7f613c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.467+0000 7f615159a640 1 --2- 172.21.15.57:0/2597764742 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f613c005240 0x7f613c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.467+0000 7f615159a640 1 -- 172.21.15.57:0/2597764742 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f6154071720 con 0x7f613c003ac0 2024-04-25T23:49:00.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.467+0000 7f615259c640 1 --2- 172.21.15.57:0/2597764742 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f613c003ac0 0x7f613c003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.467+0000 7f615159a640 1 -- 172.21.15.57:0/2597764742 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f613c00c150 con 0x7f613c005240 2024-04-25T23:49:00.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.467+0000 7f615159a640 1 -- 172.21.15.57:0/2597764742 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f613c00c350 con 0x7f613c004810 2024-04-25T23:49:00.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.467+0000 7f3c1a177640 1 -- 172.21.15.57:0/1709096961 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f3c1406efb0 msgr2=0x7f3c14143fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:49:00.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.467+0000 7f6151d9b640 1 --2- 172.21.15.57:0/2597764742 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f615413bcb0 0x7f615413e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T23:49:00.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.467+0000 7f6152d9d640 1 --2- 172.21.15.57:0/2597764742 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f613c005240 0x7f613c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.467+0000 7f6151d9b640 1 --2- 172.21.15.57:0/2597764742 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f613c004810 0x7f613c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.471+0000 7f615259c640 1 -- 172.21.15.57:0/2597764742 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f613c004810 msgr2=0x7f613c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:49:00.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.471+0000 7f615259c640 1 --2- 172.21.15.57:0/2597764742 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f613c004810 0x7f613c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.471+0000 7f615259c640 1 -- 172.21.15.57:0/2597764742 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f613c005240 msgr2=0x7f613c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:49:00.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.471+0000 7f615259c640 1 --2- 172.21.15.57:0/2597764742 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f613c005240 0x7f613c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.471+0000 7f615259c640 1 -- 172.21.15.57:0/2597764742 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f613c003530 con 0x7f613c003ac0 2024-04-25T23:49:00.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.471+0000 7f6151d9b640 1 --2- 172.21.15.57:0/2597764742 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f613c004810 0x7f613c004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T23:49:00.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.471+0000 7f3c1a177640 1 --2- 172.21.15.57:0/1709096961 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f3c1406efb0 0x7f3c14143fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.471+0000 7f3c1a177640 1 -- 172.21.15.57:0/1709096961 >> v1:172.21.15.80:6789/0 conn(0x7f3c14144500 legacy=0x7f3c141470a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.471+0000 7f3c1a177640 1 -- 172.21.15.57:0/1709096961 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c1406ff80 con 0x7f3c1406f390 2024-04-25T23:49:00.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.471+0000 7f3c19976640 1 --2- 172.21.15.57:0/1709096961 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f3c1406efb0 0x7f3c14143fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T23:49:00.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.471+0000 7fa6f5795640 1 Processor -- start 2024-04-25T23:49:00.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.471+0000 7fa6f5795640 1 -- start start 2024-04-25T23:49:00.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.471+0000 7fa6f5795640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fa6f006f0a0 0x7fa6f013bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.471+0000 7fa6f5795640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fa6f013c2a0 0x7fa6f013eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.471+0000 7fa6f5795640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6f0072d20 con 0x7fa6f0071cd0 2024-04-25T23:49:00.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.471+0000 7fa6f5795640 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 -- 0x7fa6f00715c0 con 0x7fa6f006f0a0 2024-04-25T23:49:00.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.471+0000 7fa6f5795640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7fa6f013ea10 con 0x7fa6f013c2a0 2024-04-25T23:49:00.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.471+0000 7fa6f4f94640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fa6f013c2a0 0x7fa6f013eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.471+0000 7fa6f4f94640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fa6f013c2a0 0x7fa6f013eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3301/0 says I am v2:172.21.15.57:47380/0 (socket says 172.21.15.57:47380) 2024-04-25T23:49:00.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.471+0000 7fa6f4f94640 1 -- 172.21.15.57:0/1417429895 learned_addr learned my addr 172.21.15.57:0/1417429895 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:49:00.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.471+0000 7f3c1a177640 1 --2- 172.21.15.57:0/1709096961 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c1406f390 0x7f3c14151490 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c009e50 tx=0x7f3c0c01c9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ea887a53a2157da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:00.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.471+0000 7fa6ef7fe640 1 --2- 172.21.15.57:0/1417429895 >> v2:172.21.15.57:3300/0 conn(0x7fa6f006f0a0 0x7fa6f013bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.471+0000 7f615259c640 1 --2- 172.21.15.57:0/2597764742 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f613c003ac0 0x7f613c003ea0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f614400d8f0 tx=0x7f614401f3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1116a123850011f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:00.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.471+0000 7fa6ef7fe640 1 -- 172.21.15.57:0/1417429895 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fa6f013c2a0 msgr2=0x7fa6f013eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:49:00.464 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.471+0000 7fa6ef7fe640 1 --2- 172.21.15.57:0/1417429895 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fa6f013c2a0 0x7fa6f013eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.464 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.471+0000 7fa6ef7fe640 1 -- 172.21.15.57:0/1417429895 >> v1:172.21.15.80:6789/0 conn(0x7fa6f0071cd0 legacy=0x7fa6f00720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:49:00.464 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.471+0000 7fa6ef7fe640 1 -- 172.21.15.57:0/1417429895 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6f013f430 con 0x7fa6f006f0a0 2024-04-25T23:49:00.464 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.471+0000 7f615159a640 1 -- 172.21.15.57:0/2597764742 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6144007050 con 0x7f613c003ac0 2024-04-25T23:49:00.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.471+0000 7f615159a640 1 -- 172.21.15.57:0/2597764742 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61440af430 con 0x7f613c003ac0 2024-04-25T23:49:00.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.471+0000 7f615159a640 1 -- 172.21.15.57:0/2597764742 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f614400d590 con 0x7f613c003ac0 2024-04-25T23:49:00.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.475+0000 7f615359e640 1 -- 172.21.15.57:0/2597764742 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f613c003ac0 msgr2=0x7f613c003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.475+0000 7f615359e640 1 --2- 172.21.15.57:0/2597764742 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f613c003ac0 0x7f613c003ea0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f614400d8f0 tx=0x7f614401f3b0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.475+0000 7f6152d9d640 1 -- 172.21.15.57:0/2597764742 reap_dead start 2024-04-25T23:49:00.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.475+0000 7f615359e640 1 -- 172.21.15.57:0/2597764742 shutdown_connections 2024-04-25T23:49:00.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.475+0000 7f615359e640 1 -- 172.21.15.57:0/2597764742 >> 172.21.15.57:0/2597764742 conn(0x7f61540b42a0 msgr2=0x7f61540b46b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:49:00.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.475+0000 7f615359e640 1 -- 172.21.15.57:0/2597764742 shutdown_connections 2024-04-25T23:49:00.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.475+0000 7f615359e640 1 -- 172.21.15.57:0/2597764742 wait complete. 2024-04-25T23:49:00.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.475+0000 7fed4c783640 1 Processor -- start 2024-04-25T23:49:00.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.475+0000 7fed4c783640 1 -- start start 2024-04-25T23:49:00.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.475+0000 7fed4c783640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fed440b53b0 0x7fed4413f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.475+0000 7fed4c783640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fed4413f850 0x7fed44145990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.475+0000 7fed4c783640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fed44145ed0 0x7fed440b2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.475+0000 7fed4c783640 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 -- 0x7fed440b5790 con 0x7fed44145ed0 2024-04-25T23:49:00.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.475+0000 7fed4c783640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fed440ab2f0 con 0x7fed4413f850 2024-04-25T23:49:00.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.475+0000 7fed4c783640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7fed440ab5d0 con 0x7fed440b53b0 2024-04-25T23:49:00.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.475+0000 7fed4a51f640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fed440b53b0 0x7fed4413f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.475+0000 7fed4a51f640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fed440b53b0 0x7fed4413f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3301/0 says I am v2:172.21.15.57:47388/0 (socket says 172.21.15.57:47388) 2024-04-25T23:49:00.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.475+0000 7fed4a51f640 1 -- 172.21.15.57:0/3482920123 learned_addr learned my addr 172.21.15.57:0/3482920123 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:49:00.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.475+0000 7fed4ad20640 1 --2- 172.21.15.57:0/3482920123 >> v2:172.21.15.57:3300/0 conn(0x7fed44145ed0 0x7fed440b2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.475+0000 7f615359e640 1 Processor -- start 2024-04-25T23:49:00.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.475+0000 7fa6ef7fe640 1 --2- 172.21.15.57:0/1417429895 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa6f006f0a0 0x7fa6f013bd60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8009e10 tx=0x7fa6e8020f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8976976a6bc59cd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:00.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.475+0000 7fed4a51f640 1 -- 172.21.15.57:0/3482920123 >> v2:172.21.15.80:3300/0 conn(0x7fed4413f850 msgr2=0x7fed44145990 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T23:49:00.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.475+0000 7fed4a51f640 1 --2- 172.21.15.57:0/3482920123 >> v2:172.21.15.80:3300/0 conn(0x7fed4413f850 0x7fed44145990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.475+0000 7f615359e640 1 -- start start 2024-04-25T23:49:00.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.475+0000 7fed4a51f640 1 -- 172.21.15.57:0/3482920123 >> v2:172.21.15.57:3300/0 conn(0x7fed44145ed0 msgr2=0x7fed440b2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:49:00.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.475+0000 7fed4a51f640 1 --2- 172.21.15.57:0/3482920123 >> v2:172.21.15.57:3300/0 conn(0x7fed44145ed0 0x7fed440b2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.475+0000 7fed4a51f640 1 -- 172.21.15.57:0/3482920123 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed440a9ec0 con 0x7fed440b53b0 2024-04-25T23:49:00.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.475+0000 7fa6eeffd640 1 -- 172.21.15.57:0/1417429895 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e80b5070 con 0x7fa6f006f0a0 2024-04-25T23:49:00.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.475+0000 7fa6eeffd640 1 -- 172.21.15.57:0/1417429895 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6e80b0600 con 0x7fa6f006f0a0 2024-04-25T23:49:00.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.475+0000 7fa6eeffd640 1 -- 172.21.15.57:0/1417429895 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e80b09d0 con 0x7fa6f006f0a0 2024-04-25T23:49:00.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.475+0000 7f615359e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f61541493f0 0x7f61541373c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.475+0000 7f615359e640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f61541497d0 0x7f6154147590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.475+0000 7f615359e640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f61541424f0 0x7f6154147ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.475+0000 7f615359e640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f615413c090 con 0x7f61541424f0 2024-04-25T23:49:00.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.475+0000 7f615359e640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f615413e960 con 0x7f61541493f0 2024-04-25T23:49:00.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.475+0000 7f615359e640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f61540727f0 con 0x7f61541497d0 2024-04-25T23:49:00.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7f6152d9d640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f61541424f0 0x7f6154147ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7f6151d9b640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f61541497d0 0x7f6154147590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7f6152d9d640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f61541424f0 0x7f6154147ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33456/0 (socket says 172.21.15.57:33456) 2024-04-25T23:49:00.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7f615259c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f61541493f0 0x7f61541373c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7f6152d9d640 1 -- 172.21.15.57:0/3862342756 learned_addr learned my addr 172.21.15.57:0/3862342756 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:49:00.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7f6151d9b640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f61541497d0 0x7f6154147590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3301/0 says I am v2:172.21.15.57:47390/0 (socket says 172.21.15.57:47390) 2024-04-25T23:49:00.470 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7fa6f5795640 1 -- 172.21.15.57:0/1417429895 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa6f006f0a0 msgr2=0x7fa6f013bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7fa6f5795640 1 --2- 172.21.15.57:0/1417429895 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa6f006f0a0 0x7fa6f013bd60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8009e10 tx=0x7fa6e8020f60 comp rx=0 tx=0).stop 2024-04-25T23:49:00.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7fa6f5795640 1 -- 172.21.15.57:0/1417429895 shutdown_connections 2024-04-25T23:49:00.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7fa6f5795640 1 --2- 172.21.15.57:0/1417429895 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fa6f013c2a0 0x7fa6f013eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7fa6f5795640 1 --2- 172.21.15.57:0/1417429895 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa6f006f0a0 0x7fa6f013bd60 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7fa6f5795640 1 -- 172.21.15.57:0/1417429895 >> 172.21.15.57:0/1417429895 conn(0x7fa6f006b210 msgr2=0x7fa6f006b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:49:00.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7f6151d9b640 1 -- 172.21.15.57:0/3862342756 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f61541493f0 msgr2=0x7f61541373c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7f6151d9b640 1 --2- 172.21.15.57:0/3862342756 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f61541493f0 0x7f61541373c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7f6151d9b640 1 -- 172.21.15.57:0/3862342756 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f61541424f0 msgr2=0x7f6154147ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.471 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7f6151d9b640 1 --2- 172.21.15.57:0/3862342756 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f61541424f0 0x7f6154147ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7f6151d9b640 1 -- 172.21.15.57:0/3862342756 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6154148c60 con 0x7f61541497d0 2024-04-25T23:49:00.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7fa6f5795640 1 -- 172.21.15.57:0/1417429895 shutdown_connections 2024-04-25T23:49:00.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7f6151d9b640 1 --2- 172.21.15.57:0/3862342756 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f61541497d0 0x7f6154147590 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f61540b1a20 tx=0x7f614800fd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:00.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7f6142ffd640 1 -- 172.21.15.57:0/3862342756 <== mon.2 v2:172.21.15.57:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6148017070 con 0x7f61541497d0 2024-04-25T23:49:00.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7fed4a51f640 1 --2- 172.21.15.57:0/3482920123 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fed440b53b0 0x7fed4413f310 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fed440b9220 tx=0x7fed3401c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6ed687d1d1b9815 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:00.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7fa6f5795640 1 -- 172.21.15.57:0/1417429895 wait complete. 2024-04-25T23:49:00.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7fa6f5795640 1 Processor -- start 2024-04-25T23:49:00.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7fed4951d640 1 -- 172.21.15.57:0/3482920123 <== mon.2 v2:172.21.15.57:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed340b0070 con 0x7fed440b53b0 2024-04-25T23:49:00.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7fed4951d640 1 -- 172.21.15.57:0/3482920123 <== mon.2 v2:172.21.15.57:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed340ab600 con 0x7fed440b53b0 2024-04-25T23:49:00.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7fed4951d640 1 -- 172.21.15.57:0/3482920123 <== mon.2 v2:172.21.15.57:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed340ab9d0 con 0x7fed440b53b0 2024-04-25T23:49:00.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7f6142ffd640 1 -- 172.21.15.57:0/3862342756 <== mon.2 v2:172.21.15.57:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6148004440 con 0x7f61541497d0 2024-04-25T23:49:00.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7f6142ffd640 1 -- 172.21.15.57:0/3862342756 <== mon.2 v2:172.21.15.57:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f614800b040 con 0x7f61541497d0 2024-04-25T23:49:00.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7f615359e640 1 -- 172.21.15.57:0/3862342756 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61541470b0 con 0x7f61541497d0 2024-04-25T23:49:00.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7f615359e640 1 -- 172.21.15.57:0/3862342756 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6154137900 con 0x7f61541497d0 2024-04-25T23:49:00.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7fed4c783640 1 -- 172.21.15.57:0/3482920123 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fed440b53b0 msgr2=0x7fed4413f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7fed4c783640 1 --2- 172.21.15.57:0/3482920123 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fed440b53b0 0x7fed4413f310 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fed440b9220 tx=0x7fed3401c400 comp rx=0 tx=0).stop 2024-04-25T23:49:00.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7fa6f5795640 1 -- start start 2024-04-25T23:49:00.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7fa6f5795640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa6f006f0a0 0x7fa6f0141e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7fa6f5795640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fa6f0071cd0 0x7fa6f0134980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7fa6f5795640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa6f013c2a0 0x7fa6f0131640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7fa6f5795640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f014bc90 con 0x7fa6f006f0a0 2024-04-25T23:49:00.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7fa6f5795640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f014bb10 con 0x7fa6f013c2a0 2024-04-25T23:49:00.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7fa6f5795640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7fa6f014be10 con 0x7fa6f0071cd0 2024-04-25T23:49:00.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7fa6effff640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa6f006f0a0 0x7fa6f0141e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7fa6effff640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa6f006f0a0 0x7fa6f0141e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33480/0 (socket says 172.21.15.57:33480) 2024-04-25T23:49:00.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7fa6effff640 1 -- 172.21.15.57:0/403657498 learned_addr learned my addr 172.21.15.57:0/403657498 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:49:00.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7fa6f4f94640 1 --2- 172.21.15.57:0/403657498 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa6f013c2a0 0x7fa6f0131640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7fa6effff640 1 -- 172.21.15.57:0/403657498 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fa6f0071cd0 msgr2=0x7fa6f0134980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7fa6effff640 1 --2- 172.21.15.57:0/403657498 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fa6f0071cd0 0x7fa6f0134980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7fa6effff640 1 -- 172.21.15.57:0/403657498 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa6f013c2a0 msgr2=0x7fa6f0131640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7fa6effff640 1 --2- 172.21.15.57:0/403657498 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa6f013c2a0 0x7fa6f0131640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7fa6effff640 1 -- 172.21.15.57:0/403657498 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6f01350e0 con 0x7fa6f006f0a0 2024-04-25T23:49:00.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7fa6effff640 1 --2- 172.21.15.57:0/403657498 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa6f006f0a0 0x7fa6f0141e60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa6e400c450 tx=0x7fa6e400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:00.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7fed4c783640 1 -- 172.21.15.57:0/3482920123 shutdown_connections 2024-04-25T23:49:00.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7fed4c783640 1 --2- 172.21.15.57:0/3482920123 >> v2:172.21.15.57:3300/0 conn(0x7fed44145ed0 0x7fed440b2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7fed4c783640 1 --2- 172.21.15.57:0/3482920123 >> v2:172.21.15.80:3300/0 conn(0x7fed4413f850 0x7fed44145990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7fed4c783640 1 --2- 172.21.15.57:0/3482920123 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fed440b53b0 0x7fed4413f310 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7fed4c783640 1 -- 172.21.15.57:0/3482920123 >> 172.21.15.57:0/3482920123 conn(0x7fed44074cf0 msgr2=0x7fed44075100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:49:00.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7fed4c783640 1 -- 172.21.15.57:0/3482920123 shutdown_connections 2024-04-25T23:49:00.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7fed4c783640 1 -- 172.21.15.57:0/3482920123 wait complete. 2024-04-25T23:49:00.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7f615359e640 1 -- 172.21.15.57:0/3862342756 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f615404f510 con 0x7f61541497d0 2024-04-25T23:49:00.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7f9933276640 1 Processor -- start 2024-04-25T23:49:00.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7f9933276640 1 -- start start 2024-04-25T23:49:00.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7f9933276640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f992c06ea10 0x7f992c08aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7f9933276640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f992c06fa80 0x7f992c08afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7f9933276640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f992c14f070 con 0x7f992c06edf0 2024-04-25T23:49:00.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7f9933276640 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 -- 0x7f992c0ae830 con 0x7f992c06ea10 2024-04-25T23:49:00.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7f9933276640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f992c08bc60 con 0x7f992c06fa80 2024-04-25T23:49:00.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7f9932274640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f992c06ea10 0x7f992c08aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7f9932274640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f992c06ea10 0x7f992c08aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33466/0 (socket says 172.21.15.57:33466) 2024-04-25T23:49:00.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7f9932274640 1 -- 172.21.15.57:0/2846258560 learned_addr learned my addr 172.21.15.57:0/2846258560 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:49:00.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.483+0000 7f9932274640 1 -- 172.21.15.57:0/2846258560 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f992c06fa80 msgr2=0x7f992c08afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:49:00.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.479+0000 7f9931a73640 1 --2- 172.21.15.57:0/2846258560 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f992c06fa80 0x7f992c08afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.483+0000 7f9932274640 1 --2- 172.21.15.57:0/2846258560 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f992c06fa80 0x7f992c08afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.483+0000 7f9932274640 1 -- 172.21.15.57:0/2846258560 >> v1:172.21.15.80:6789/0 conn(0x7f992c06edf0 legacy=0x7f992c14f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.483+0000 7f9932274640 1 -- 172.21.15.57:0/2846258560 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f992c14fc80 con 0x7f992c06ea10 2024-04-25T23:49:00.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.483+0000 7fed4c783640 1 Processor -- start 2024-04-25T23:49:00.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.483+0000 7fed4c783640 1 -- start start 2024-04-25T23:49:00.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.483+0000 7f6142ffd640 1 -- 172.21.15.57:0/3862342756 <== mon.2 v2:172.21.15.57:3301/0 4 ==== mgrmap(e 4) ==== 87354+0+0 (secure 0 0 0) 0x7f614801a070 con 0x7f61541497d0 2024-04-25T23:49:00.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.483+0000 7f6142ffd640 1 --2- 172.21.15.57:0/3862342756 >> v2:172.21.15.80:6816/2493457677 conn(0x7f61340677f0 0x7f6134069cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.483+0000 7f6142ffd640 1 -- 172.21.15.57:0/3862342756 <== mon.2 v2:172.21.15.57:3301/0 5 ==== osd_map(17..17 src has 1..17) ==== 4090+0+0 (secure 0 0 0) 0x7f6148085600 con 0x7f61541497d0 2024-04-25T23:49:00.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.483+0000 7fed4c783640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fed440b53b0 0x7fed44146f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.483+0000 7f615259c640 1 --2- 172.21.15.57:0/3862342756 >> v2:172.21.15.80:6816/2493457677 conn(0x7f61340677f0 0x7f6134069cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.483+0000 7fed4c783640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fed4413f850 0x7fed44140cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.483+0000 7fed4c783640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fed44145ed0 0x7fed44141200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.483+0000 7fed4c783640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fed4414eeb0 con 0x7fed440b53b0 2024-04-25T23:49:00.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.483+0000 7fed4c783640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fed4414ed30 con 0x7fed4413f850 2024-04-25T23:49:00.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.483+0000 7fed4c783640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7fed4414f030 con 0x7fed44145ed0 2024-04-25T23:49:00.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.483+0000 7fa6ecff9640 1 -- 172.21.15.57:0/403657498 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e4016020 con 0x7fa6f006f0a0 2024-04-25T23:49:00.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.483+0000 7fa6f5795640 1 -- 172.21.15.57:0/403657498 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6f0134050 con 0x7fa6f006f0a0 2024-04-25T23:49:00.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.483+0000 7fa6f5795640 1 -- 172.21.15.57:0/403657498 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6f0131d50 con 0x7fa6f006f0a0 2024-04-25T23:49:00.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.483+0000 7fa6ecff9640 1 -- 172.21.15.57:0/403657498 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6e40052d0 con 0x7fa6f006f0a0 2024-04-25T23:49:00.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.483+0000 7fa6ecff9640 1 -- 172.21.15.57:0/403657498 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e4007240 con 0x7fa6f006f0a0 2024-04-25T23:49:00.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.483+0000 7fed4a51f640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fed440b53b0 0x7fed44146f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.483+0000 7fed4a51f640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fed440b53b0 0x7fed44146f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33484/0 (socket says 172.21.15.57:33484) 2024-04-25T23:49:00.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.483+0000 7fed4a51f640 1 -- 172.21.15.57:0/2497365725 learned_addr learned my addr 172.21.15.57:0/2497365725 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:49:00.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.483+0000 7f615259c640 1 --2- 172.21.15.57:0/3862342756 >> v2:172.21.15.80:6816/2493457677 conn(0x7f61340677f0 0x7f6134069cb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f61440af000 tx=0x7f61440a9000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:00.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.483+0000 7fed4a51f640 1 -- 172.21.15.57:0/2497365725 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fed44145ed0 msgr2=0x7fed44141200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.483+0000 7fed4a51f640 1 --2- 172.21.15.57:0/2497365725 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fed44145ed0 0x7fed44141200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.483+0000 7fed4a51f640 1 -- 172.21.15.57:0/2497365725 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fed4413f850 msgr2=0x7fed44140cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.483+0000 7fed4a51f640 1 --2- 172.21.15.57:0/2497365725 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fed4413f850 0x7fed44140cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.483+0000 7fed4a51f640 1 -- 172.21.15.57:0/2497365725 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed44148d80 con 0x7fed440b53b0 2024-04-25T23:49:00.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.483+0000 7fed4a51f640 1 --2- 172.21.15.57:0/2497365725 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fed440b53b0 0x7fed44146f40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fed3401a030 tx=0x7fed340191a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:00.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.483+0000 7fa6ecff9640 1 -- 172.21.15.57:0/403657498 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 4) ==== 87354+0+0 (secure 0 0 0) 0x7fa6e40073e0 con 0x7fa6f006f0a0 2024-04-25T23:49:00.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.483+0000 7fed3affd640 1 -- 172.21.15.57:0/2497365725 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed340b0070 con 0x7fed440b53b0 2024-04-25T23:49:00.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.483+0000 7fed4c783640 1 -- 172.21.15.57:0/2497365725 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed44141740 con 0x7fed440b53b0 2024-04-25T23:49:00.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.483+0000 7fed4c783640 1 -- 172.21.15.57:0/2497365725 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed44149060 con 0x7fed440b53b0 2024-04-25T23:49:00.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.483+0000 7f9931a73640 1 --2- 172.21.15.57:0/2846258560 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f992c06fa80 0x7f992c08afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T23:49:00.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.487+0000 7f9932274640 1 --2- 172.21.15.57:0/2846258560 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f992c06ea10 0x7f992c08aa70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f991c0088f0 tx=0x7f991c01c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7257d117fd637678 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:00.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.483+0000 7f67d109e640 1 Processor -- start 2024-04-25T23:49:00.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.487+0000 7f67d109e640 1 -- start start 2024-04-25T23:49:00.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.487+0000 7fed3affd640 1 -- 172.21.15.57:0/2497365725 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed340b1040 con 0x7fed440b53b0 2024-04-25T23:49:00.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.487+0000 7fed3affd640 1 -- 172.21.15.57:0/2497365725 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed340aa7d0 con 0x7fed440b53b0 2024-04-25T23:49:00.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.487+0000 7f9931272640 1 -- 172.21.15.57:0/2846258560 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f991c0a5950 con 0x7f992c06ea10 2024-04-25T23:49:00.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.487+0000 7f9931272640 1 -- 172.21.15.57:0/2846258560 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f991c00ae50 con 0x7f992c06ea10 2024-04-25T23:49:00.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.487+0000 7fed4c783640 1 -- 172.21.15.57:0/2497365725 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed2c005090 con 0x7fed440b53b0 2024-04-25T23:49:00.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.487+0000 7fa6ecff9640 1 --2- 172.21.15.57:0/403657498 >> v2:172.21.15.80:6816/2493457677 conn(0x7fa6d8067950 0x7fa6d8069e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.487+0000 7fa6f5795640 1 -- 172.21.15.57:0/403657498 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6d0005090 con 0x7fa6f006f0a0 2024-04-25T23:49:00.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.487+0000 7fa6ecff9640 1 -- 172.21.15.57:0/403657498 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4090+0+0 (secure 0 0 0) 0x7fa6e4084ec0 con 0x7fa6f006f0a0 2024-04-25T23:49:00.489 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.487+0000 7fa6ef7fe640 1 --2- 172.21.15.57:0/403657498 >> v2:172.21.15.80:6816/2493457677 conn(0x7fa6d8067950 0x7fa6d8069e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.487+0000 7f9931272640 1 -- 172.21.15.57:0/2846258560 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f991c0a5cf0 con 0x7f992c06ea10 2024-04-25T23:49:00.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.487+0000 7f9933276640 1 -- 172.21.15.57:0/2846258560 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f992c06ea10 msgr2=0x7f992c08aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.487+0000 7f9933276640 1 --2- 172.21.15.57:0/2846258560 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f992c06ea10 0x7f992c08aa70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f991c0088f0 tx=0x7f991c01c890 comp rx=0 tx=0).stop 2024-04-25T23:49:00.490 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.487+0000 7fed3affd640 1 -- 172.21.15.57:0/2497365725 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 4) ==== 87354+0+0 (secure 0 0 0) 0x7fed340aa970 con 0x7fed440b53b0 2024-04-25T23:49:00.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.487+0000 7f67d109e640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f67cc070790 0x7f67cc142560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.487+0000 7f67d109e640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f67cc142aa0 0x7f67cc072cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.487+0000 7f67d109e640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67cc073700 con 0x7f67cc070b70 2024-04-25T23:49:00.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.487+0000 7f67d109e640 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 -- 0x7f67cc141de0 con 0x7f67cc070790 2024-04-25T23:49:00.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.487+0000 7f67d109e640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f67cc06fff0 con 0x7f67cc142aa0 2024-04-25T23:49:00.491 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.487+0000 7fed3affd640 1 --2- 172.21.15.57:0/2497365725 >> v2:172.21.15.80:6816/2493457677 conn(0x7fed280678c0 0x7fed28069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.487+0000 7f9933276640 1 -- 172.21.15.57:0/2846258560 shutdown_connections 2024-04-25T23:49:00.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.487+0000 7f9933276640 1 --2- 172.21.15.57:0/2846258560 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f992c06fa80 0x7f992c08afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.487+0000 7f9933276640 1 --2- 172.21.15.57:0/2846258560 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f992c06ea10 0x7f992c08aa70 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.487+0000 7f9933276640 1 -- 172.21.15.57:0/2846258560 >> 172.21.15.57:0/2846258560 conn(0x7f992c0752a0 msgr2=0x7f992c0b3e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:49:00.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.487+0000 7f67d089d640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f67cc142aa0 0x7f67cc072cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.487+0000 7f67d089d640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f67cc142aa0 0x7f67cc072cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3301/0 says I am v2:172.21.15.57:47418/0 (socket says 172.21.15.57:47418) 2024-04-25T23:49:00.492 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.487+0000 7f67d089d640 1 -- 172.21.15.57:0/1334609179 learned_addr learned my addr 172.21.15.57:0/1334609179 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:49:00.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.487+0000 7fed3affd640 1 -- 172.21.15.57:0/2497365725 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4090+0+0 (secure 0 0 0) 0x7fed3411fe10 con 0x7fed440b53b0 2024-04-25T23:49:00.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.487+0000 7fed49d1e640 1 --2- 172.21.15.57:0/2497365725 >> v2:172.21.15.80:6816/2493457677 conn(0x7fed280678c0 0x7fed28069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.487+0000 7f67d089d640 1 -- 172.21.15.57:0/1334609179 >> v2:172.21.15.57:3300/0 conn(0x7f67cc070790 msgr2=0x7f67cc142560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:49:00.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.487+0000 7f67d089d640 1 --2- 172.21.15.57:0/1334609179 >> v2:172.21.15.57:3300/0 conn(0x7f67cc070790 0x7f67cc142560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.487+0000 7f67d089d640 1 -- 172.21.15.57:0/1334609179 >> v1:172.21.15.57:6789/0 conn(0x7f67cc070b70 legacy=0x7f67cc0725c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T23:49:00.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.487+0000 7f67d089d640 1 -- 172.21.15.57:0/1334609179 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67cc0aeb00 con 0x7f67cc142aa0 2024-04-25T23:49:00.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.487+0000 7fa6ef7fe640 1 --2- 172.21.15.57:0/403657498 >> v2:172.21.15.80:6816/2493457677 conn(0x7fa6d8067950 0x7fa6d8069e10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa6e80b2000 tx=0x7fa6e8004ac0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:00.494 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.487+0000 7fed49d1e640 1 --2- 172.21.15.57:0/2497365725 >> v2:172.21.15.80:6816/2493457677 conn(0x7fed280678c0 0x7fed28069d80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fed44147480 tx=0x7fed40006c50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:00.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.491+0000 7f8c0e248640 1 Processor -- start 2024-04-25T23:49:00.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.491+0000 7f8c0e248640 1 -- start start 2024-04-25T23:49:00.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.491+0000 7f8c0e248640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f8c0813a370 0x7f8c080b0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.491+0000 7f8c0e248640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c08073240 con 0x7f8c080724c0 2024-04-25T23:49:00.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.491+0000 7f8c0e248640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c08073520 con 0x7f8c08071250 2024-04-25T23:49:00.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.491+0000 7f8c0e248640 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 -- 0x7f8c08071630 con 0x7f8c0813a370 2024-04-25T23:49:00.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.491+0000 7f8c0ca45640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f8c080724c0 0x7f8c08139c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:49662/0 (socket says 172.21.15.57:49662) 2024-04-25T23:49:00.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.491+0000 7f8c0ca45640 1 -- 172.21.15.57:0/2999878836 learned_addr learned my addr 172.21.15.57:0/2999878836 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-04-25T23:49:00.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.491+0000 7f8c0da47640 1 --2- 172.21.15.57:0/2999878836 >> v2:172.21.15.57:3300/0 conn(0x7f8c0813a370 0x7f8c080b0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.495 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.491+0000 7f9933276640 1 -- 172.21.15.57:0/2846258560 shutdown_connections 2024-04-25T23:49:00.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.491+0000 7f8c0da47640 1 -- 172.21.15.57:0/2999878836 >> v1:172.21.15.80:6789/0 conn(0x7f8c08071250 legacy=0x7f8c08140a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.491+0000 7f8c0da47640 1 -- 172.21.15.57:0/2999878836 >> v1:172.21.15.57:6789/0 conn(0x7f8c080724c0 legacy=0x7f8c08139c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.491+0000 7f8c0da47640 1 -- 172.21.15.57:0/2999878836 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c080717b0 con 0x7f8c0813a370 2024-04-25T23:49:00.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.491+0000 7f67d089d640 1 --2- 172.21.15.57:0/1334609179 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f67cc142aa0 0x7f67cc072cd0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f67c400d150 tx=0x7f67c401eab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a67baa9568266e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:00.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.491+0000 7f9933276640 1 -- 172.21.15.57:0/2846258560 wait complete. 2024-04-25T23:49:00.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.491+0000 7f67caffd640 1 -- 172.21.15.57:0/1334609179 <== mon.2 v2:172.21.15.57:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c40ad070 con 0x7f67cc142aa0 2024-04-25T23:49:00.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.491+0000 7f6142ffd640 1 -- 172.21.15.57:0/3862342756 <== mon.2 v2:172.21.15.57:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f61480562b0 con 0x7f61541497d0 2024-04-25T23:49:00.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.491+0000 7f67caffd640 1 -- 172.21.15.57:0/1334609179 <== mon.2 v2:172.21.15.57:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67c40a79d0 con 0x7f67cc142aa0 2024-04-25T23:49:00.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.491+0000 7f9933276640 1 Processor -- start 2024-04-25T23:49:00.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.491+0000 7f67caffd640 1 -- 172.21.15.57:0/1334609179 <== mon.2 v2:172.21.15.57:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c40a7da0 con 0x7f67cc142aa0 2024-04-25T23:49:00.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.491+0000 7fed3affd640 1 -- 172.21.15.57:0/2497365725 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fed44141740 con 0x7fed440b53b0 2024-04-25T23:49:00.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.491+0000 7f9933276640 1 -- start start 2024-04-25T23:49:00.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.495+0000 7f67d109e640 1 -- 172.21.15.57:0/1334609179 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f67cc142aa0 msgr2=0x7f67cc072cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.495+0000 7f67d109e640 1 --2- 172.21.15.57:0/1334609179 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f67cc142aa0 0x7f67cc072cd0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f67c400d150 tx=0x7f67c401eab0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.495+0000 7f67d109e640 1 -- 172.21.15.57:0/1334609179 shutdown_connections 2024-04-25T23:49:00.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.495+0000 7f67d109e640 1 --2- 172.21.15.57:0/1334609179 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f67cc142aa0 0x7f67cc072cd0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.495+0000 7f67d109e640 1 --2- 172.21.15.57:0/1334609179 >> v2:172.21.15.57:3300/0 conn(0x7f67cc070790 0x7f67cc142560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.495+0000 7f67d109e640 1 -- 172.21.15.57:0/1334609179 >> 172.21.15.57:0/1334609179 conn(0x7f67cc06b3e0 msgr2=0x7f67cc06b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:49:00.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.495+0000 7f8c0da47640 1 --2- 172.21.15.57:0/2999878836 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c0813a370 0x7f8c080b0bb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8c00009d90 tx=0x7f8c0001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44598d4c153208ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:00.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.495+0000 7f9933276640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f992c06ea10 0x7f992c0a9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.495+0000 7f9933276640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f992c06edf0 0x7f992c14c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.495+0000 7f9933276640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f992c06fa80 0x7f992c1eaa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.495+0000 7f9933276640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f992c15c450 con 0x7f992c06fa80 2024-04-25T23:49:00.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.495+0000 7f9933276640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f992c15c2d0 con 0x7f992c06edf0 2024-04-25T23:49:00.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.495+0000 7f9933276640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f992c15c5d0 con 0x7f992c06ea10 2024-04-25T23:49:00.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.495+0000 7f9932a75640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f992c06fa80 0x7f992c1eaa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.495+0000 7f9932a75640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f992c06fa80 0x7f992c1eaa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33516/0 (socket says 172.21.15.57:33516) 2024-04-25T23:49:00.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.495+0000 7f9932a75640 1 -- 172.21.15.57:0/503217135 learned_addr learned my addr 172.21.15.57:0/503217135 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:49:00.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.495+0000 7f9932274640 1 --2- 172.21.15.57:0/503217135 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f992c06ea10 0x7f992c0a9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.495+0000 7f9931a73640 1 --2- 172.21.15.57:0/503217135 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f992c06edf0 0x7f992c14c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.495+0000 7f9932a75640 1 -- 172.21.15.57:0/503217135 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f992c06ea10 msgr2=0x7f992c0a9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.495+0000 7f9932a75640 1 --2- 172.21.15.57:0/503217135 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f992c06ea10 0x7f992c0a9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.495+0000 7f9932a75640 1 -- 172.21.15.57:0/503217135 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f992c06edf0 msgr2=0x7f992c14c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.495+0000 7f9932a75640 1 --2- 172.21.15.57:0/503217135 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f992c06edf0 0x7f992c14c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.495+0000 7f9932a75640 1 -- 172.21.15.57:0/503217135 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f992c0aae90 con 0x7f992c06fa80 2024-04-25T23:49:00.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.495+0000 7f9932a75640 1 --2- 172.21.15.57:0/503217135 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f992c06fa80 0x7f992c1eaa00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f992400c710 tx=0x7f992400cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:00.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.495+0000 7f67d109e640 1 -- 172.21.15.57:0/1334609179 shutdown_connections 2024-04-25T23:49:00.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.495+0000 7f67d109e640 1 -- 172.21.15.57:0/1334609179 wait complete. 2024-04-25T23:49:00.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.499+0000 7f67d109e640 1 Processor -- start 2024-04-25T23:49:00.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.499+0000 7f67d109e640 1 -- start start 2024-04-25T23:49:00.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.499+0000 7f9922ffd640 1 -- 172.21.15.57:0/503217135 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9924016020 con 0x7f992c06fa80 2024-04-25T23:49:00.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.499+0000 7fa6ecff9640 1 -- 172.21.15.57:0/403657498 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa6e4020020 con 0x7fa6f006f0a0 2024-04-25T23:49:00.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.499+0000 7f8bfffff640 1 -- 172.21.15.57:0/2999878836 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c000b2070 con 0x7f8c0813a370 2024-04-25T23:49:00.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.499+0000 7f8bfffff640 1 -- 172.21.15.57:0/2999878836 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c000a5940 con 0x7f8c0813a370 2024-04-25T23:49:00.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.499+0000 7f8bfffff640 1 -- 172.21.15.57:0/2999878836 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c000a5cf0 con 0x7f8c0813a370 2024-04-25T23:49:00.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.499+0000 7f9933276640 1 -- 172.21.15.57:0/503217135 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f992c0ab100 con 0x7f992c06fa80 2024-04-25T23:49:00.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.499+0000 7f9922ffd640 1 -- 172.21.15.57:0/503217135 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f992400ce20 con 0x7f992c06fa80 2024-04-25T23:49:00.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.499+0000 7f9922ffd640 1 -- 172.21.15.57:0/503217135 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9924003980 con 0x7f992c06fa80 2024-04-25T23:49:00.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.499+0000 7f8c0e248640 1 -- 172.21.15.57:0/2999878836 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c0813a370 msgr2=0x7f8c080b0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.499+0000 7f8c0e248640 1 --2- 172.21.15.57:0/2999878836 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c0813a370 0x7f8c080b0bb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8c00009d90 tx=0x7f8c0001c970 comp rx=0 tx=0).stop 2024-04-25T23:49:00.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.499+0000 7f8c0e248640 1 -- 172.21.15.57:0/2999878836 shutdown_connections 2024-04-25T23:49:00.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.499+0000 7f8c0e248640 1 --2- 172.21.15.57:0/2999878836 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c0813a370 0x7f8c080b0bb0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.499+0000 7f8c0e248640 1 -- 172.21.15.57:0/2999878836 >> 172.21.15.57:0/2999878836 conn(0x7f8c080b62c0 msgr2=0x7f8c080b66d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:49:00.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.499+0000 7f8c0e248640 1 -- 172.21.15.57:0/2999878836 shutdown_connections 2024-04-25T23:49:00.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.499+0000 7f8c0e248640 1 -- 172.21.15.57:0/2999878836 wait complete. 2024-04-25T23:49:00.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.499+0000 7f9933276640 1 -- 172.21.15.57:0/503217135 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f992c1eaf40 con 0x7f992c06fa80 2024-04-25T23:49:00.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.503+0000 7f67d109e640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f67cc070790 0x7f67cc1372d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.503+0000 7f8c0e248640 1 Processor -- start 2024-04-25T23:49:00.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.503+0000 7f9920ff9640 1 -- 172.21.15.57:0/503217135 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f992c04f510 con 0x7f992c06fa80 2024-04-25T23:49:00.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.503+0000 7f67d109e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67cc070b70 0x7f67cc13c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.503+0000 7f67d109e640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f67cc142aa0 0x7f67cc13dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.503+0000 7f67d109e640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f67cc14bb80 con 0x7f67cc070790 2024-04-25T23:49:00.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.503+0000 7f67d109e640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f67cc0aed90 con 0x7f67cc070b70 2024-04-25T23:49:00.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.503+0000 7f67d109e640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f67cc14bd00 con 0x7f67cc142aa0 2024-04-25T23:49:00.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.503+0000 7f67d089d640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f67cc142aa0 0x7f67cc13dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.503+0000 7f67d089d640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f67cc142aa0 0x7f67cc13dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3301/0 says I am v2:172.21.15.57:47440/0 (socket says 172.21.15.57:47440) 2024-04-25T23:49:00.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.503+0000 7f67d089d640 1 -- 172.21.15.57:0/839849140 learned_addr learned my addr 172.21.15.57:0/839849140 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:49:00.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.503+0000 7f9922ffd640 1 -- 172.21.15.57:0/503217135 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 4) ==== 87354+0+0 (secure 0 0 0) 0x7f9924003b20 con 0x7f992c06fa80 2024-04-25T23:49:00.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.503+0000 7f9922ffd640 1 --2- 172.21.15.57:0/503217135 >> v2:172.21.15.80:6816/2493457677 conn(0x7f99100677f0 0x7f9910069cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.503+0000 7f8c0e248640 1 -- start start 2024-04-25T23:49:00.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.503+0000 7f8c0e248640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8c08071250 0x7f8c0813ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.503+0000 7f8c0e248640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c080724c0 0x7f8c081dbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.503+0000 7f8c0e248640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f8c0813a370 0x7f8c0812f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.503+0000 7f8c0e248640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0814ea80 con 0x7f8c080724c0 2024-04-25T23:49:00.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.503+0000 7f8c0e248640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0814e900 con 0x7f8c08071250 2024-04-25T23:49:00.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.503+0000 7f8c0e248640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f8c0814ec00 con 0x7f8c0813a370 2024-04-25T23:49:00.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.503+0000 7f9932274640 1 --2- 172.21.15.57:0/503217135 >> v2:172.21.15.80:6816/2493457677 conn(0x7f99100677f0 0x7f9910069cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.503+0000 7f9932274640 1 --2- 172.21.15.57:0/503217135 >> v2:172.21.15.80:6816/2493457677 conn(0x7f99100677f0 0x7f9910069cb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f991c0ae000 tx=0x7f991c0a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:00.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.503+0000 7f67cb7fe640 1 --2- 172.21.15.57:0/839849140 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67cc070b70 0x7f67cc13c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.503+0000 7f67d089d640 1 -- 172.21.15.57:0/839849140 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67cc070b70 msgr2=0x7f67cc13c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.503+0000 7f67d089d640 1 --2- 172.21.15.57:0/839849140 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67cc070b70 0x7f67cc13c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.503+0000 7f67d089d640 1 -- 172.21.15.57:0/839849140 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f67cc070790 msgr2=0x7f67cc1372d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.503+0000 7f67d089d640 1 --2- 172.21.15.57:0/839849140 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f67cc070790 0x7f67cc1372d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.503+0000 7f67d089d640 1 -- 172.21.15.57:0/839849140 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67cc0a9db0 con 0x7f67cc142aa0 2024-04-25T23:49:00.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.503+0000 7f67d089d640 1 --2- 172.21.15.57:0/839849140 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f67cc142aa0 0x7f67cc13dc50 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f67c401d160 tx=0x7f67c401b850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:00.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.503+0000 7f8c0d246640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8c08071250 0x7f8c0813ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.503+0000 7f8c0d246640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8c08071250 0x7f8c0813ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.57:54320/0 (socket says 172.21.15.57:54320) 2024-04-25T23:49:00.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.503+0000 7f8c0d246640 1 -- 172.21.15.57:0/188466202 learned_addr learned my addr 172.21.15.57:0/188466202 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:49:00.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.503+0000 7f8c0da47640 1 --2- 172.21.15.57:0/188466202 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f8c0813a370 0x7f8c0812f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.510 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.503+0000 7f8c0d246640 1 -- 172.21.15.57:0/188466202 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f8c0813a370 msgr2=0x7f8c0812f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.503+0000 7f8c0d246640 1 --2- 172.21.15.57:0/188466202 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f8c0813a370 0x7f8c0812f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.503+0000 7f8c0d246640 1 -- 172.21.15.57:0/188466202 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c080724c0 msgr2=0x7f8c081dbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.503+0000 7f8c0d246640 1 --2- 172.21.15.57:0/188466202 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c080724c0 0x7f8c081dbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.503+0000 7f8c0d246640 1 -- 172.21.15.57:0/188466202 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c0813ba50 con 0x7f8c08071250 2024-04-25T23:49:00.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.503+0000 7f8c0d246640 1 --2- 172.21.15.57:0/188466202 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8c08071250 0x7f8c0813ac40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8c0400c500 tx=0x7f8c0400c9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:00.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.503+0000 7f67c8ff9640 1 -- 172.21.15.57:0/839849140 <== mon.2 v2:172.21.15.57:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c40c0020 con 0x7f67cc142aa0 2024-04-25T23:49:00.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.503+0000 7f67d109e640 1 -- 172.21.15.57:0/839849140 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67cc13e190 con 0x7f67cc142aa0 2024-04-25T23:49:00.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.503+0000 7f67d109e640 1 -- 172.21.15.57:0/839849140 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f67cc13ecc0 con 0x7f67cc142aa0 2024-04-25T23:49:00.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.507+0000 7f9922ffd640 1 -- 172.21.15.57:0/503217135 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4090+0+0 (secure 0 0 0) 0x7f9924083740 con 0x7f992c06fa80 2024-04-25T23:49:00.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.507+0000 7f67c8ff9640 1 -- 172.21.15.57:0/839849140 <== mon.2 v2:172.21.15.57:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67c40a79d0 con 0x7f67cc142aa0 2024-04-25T23:49:00.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.507+0000 7f67d109e640 1 -- 172.21.15.57:0/839849140 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67ac005090 con 0x7f67cc142aa0 2024-04-25T23:49:00.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.507+0000 7f67c8ff9640 1 -- 172.21.15.57:0/839849140 <== mon.2 v2:172.21.15.57:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c40ad040 con 0x7f67cc142aa0 2024-04-25T23:49:00.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.507+0000 7f67c8ff9640 1 -- 172.21.15.57:0/839849140 <== mon.2 v2:172.21.15.57:3301/0 4 ==== mgrmap(e 4) ==== 87354+0+0 (secure 0 0 0) 0x7f67c40c3020 con 0x7f67cc142aa0 2024-04-25T23:49:00.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.511+0000 7f67c8ff9640 1 --2- 172.21.15.57:0/839849140 >> v2:172.21.15.80:6816/2493457677 conn(0x7f67b80677f0 0x7f67b8069cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.512 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.511+0000 7f67c8ff9640 1 -- 172.21.15.57:0/839849140 <== mon.2 v2:172.21.15.57:3301/0 5 ==== osd_map(17..17 src has 1..17) ==== 4090+0+0 (secure 0 0 0) 0x7f67c4123010 con 0x7f67cc142aa0 2024-04-25T23:49:00.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.511+0000 7f9922ffd640 1 -- 172.21.15.57:0/503217135 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f9924055ca0 con 0x7f992c06fa80 2024-04-25T23:49:00.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.511+0000 7f67cbfff640 1 --2- 172.21.15.57:0/839849140 >> v2:172.21.15.80:6816/2493457677 conn(0x7f67b80677f0 0x7f67b8069cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.511+0000 7f67cbfff640 1 --2- 172.21.15.57:0/839849140 >> v2:172.21.15.80:6816/2493457677 conn(0x7f67b80677f0 0x7f67b8069cb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f67bc009730 tx=0x7f67bc006ca0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:00.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.519+0000 7f67c8ff9640 1 -- 172.21.15.57:0/839849140 <== mon.2 v2:172.21.15.57:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f67c40ca030 con 0x7f67cc142aa0 2024-04-25T23:49:00.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.555+0000 7f3c18974640 1 -- 172.21.15.57:0/1709096961 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c0a5640 con 0x7f3c1406f390 2024-04-25T23:49:00.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.555+0000 7f8bfdffb640 1 -- 172.21.15.57:0/188466202 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c04016020 con 0x7f8c08071250 2024-04-25T23:49:00.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.555+0000 7f8c0e248640 1 -- 172.21.15.57:0/188466202 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c0813bd30 con 0x7f8c08071250 2024-04-25T23:49:00.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.555+0000 7f3c1a177640 1 -- 172.21.15.57:0/1709096961 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c1406f390 msgr2=0x7f3c14151490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-25T23:49:00.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.555+0000 7f3c1a177640 1 -- 172.21.15.57:0/1709096961 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c1406f390 msgr2=0x7f3c14151490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T23:49:00.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.555+0000 7f3c1a177640 1 --2- 172.21.15.57:0/1709096961 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c1406f390 0x7f3c14151490 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c009e50 tx=0x7f3c0c01c9a0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T23:49:00.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.555+0000 7f3c1a177640 1 --2- 172.21.15.57:0/1709096961 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c1406f390 0x7f3c14151490 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c009e50 tx=0x7f3c0c01c9a0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.555+0000 7f3c18974640 1 -- 172.21.15.57:0/1709096961 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c1406f390 msgr2=0x7f3c14151490 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T23:49:00.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.555+0000 7f3c18974640 1 --2- 172.21.15.57:0/1709096961 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c1406f390 0x7f3c14151490 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.555+0000 7f3c18974640 1 --2- 172.21.15.57:0/1709096961 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c00003710 0x7f3c00005b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.555+0000 7f3c18974640 1 --2- 172.21.15.57:0/1709096961 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f3c1406efb0 0x7f3c000083b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.555+0000 7f3c18974640 1 --2- 172.21.15.57:0/1709096961 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c00008b50 0x7f3c0000afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.555+0000 7f3c18974640 1 -- 172.21.15.57:0/1709096961 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0000b8f0 con 0x7f3c00003710 2024-04-25T23:49:00.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.555+0000 7f3c18974640 1 -- 172.21.15.57:0/1709096961 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0000baa0 con 0x7f3c00008b50 2024-04-25T23:49:00.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.555+0000 7f3c18974640 1 -- 172.21.15.57:0/1709096961 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f3c0000bca0 con 0x7f3c1406efb0 2024-04-25T23:49:00.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.555+0000 7f3c1a177640 1 --2- 172.21.15.57:0/1709096961 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c00008b50 0x7f3c0000afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.555+0000 7f8c0e248640 1 -- 172.21.15.57:0/188466202 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c081dca30 con 0x7f8c08071250 2024-04-25T23:49:00.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.559+0000 7f8c0e248640 1 -- 172.21.15.57:0/188466202 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c0804f510 con 0x7f8c08071250 2024-04-25T23:49:00.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.559+0000 7f3c1a177640 1 -- 172.21.15.57:0/1709096961 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f3c1406efb0 msgr2=0x7f3c000083b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:49:00.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.559+0000 7f3c1a177640 1 --2- 172.21.15.57:0/1709096961 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f3c1406efb0 0x7f3c000083b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.559+0000 7f3c1a177640 1 -- 172.21.15.57:0/1709096961 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c00003710 msgr2=0x7f3c00005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:49:00.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.559+0000 7f3c1a177640 1 --2- 172.21.15.57:0/1709096961 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c00003710 0x7f3c00005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.559+0000 7f3c1a177640 1 -- 172.21.15.57:0/1709096961 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=2+}) -- 0x7f3c0000bf60 con 0x7f3c00008b50 2024-04-25T23:49:00.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.559+0000 7f3c1a177640 1 --2- 172.21.15.57:0/1709096961 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c00008b50 0x7f3c0000afe0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c0ab000 tx=0x7f3c0c009810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5748b2ffee9e1e21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:00.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.559+0000 7f3c18974640 1 -- 172.21.15.57:0/1709096961 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c0ba070 con 0x7f3c00008b50 2024-04-25T23:49:00.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.559+0000 7f8bfdffb640 1 -- 172.21.15.57:0/188466202 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c04005220 con 0x7f8c08071250 2024-04-25T23:49:00.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.559+0000 7f8bfdffb640 1 -- 172.21.15.57:0/188466202 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c04007030 con 0x7f8c08071250 2024-04-25T23:49:00.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.559+0000 7f8bfdffb640 1 -- 172.21.15.57:0/188466202 <== mon.1 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 4) ==== 87354+0+0 (secure 0 0 0) 0x7f8c040071d0 con 0x7f8c08071250 2024-04-25T23:49:00.549 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.559+0000 7f8bfdffb640 1 --2- 172.21.15.57:0/188466202 >> v2:172.21.15.80:6816/2493457677 conn(0x7f8bec0677f0 0x7f8bec069cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.549 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.559+0000 7f8bfdffb640 1 -- 172.21.15.57:0/188466202 <== mon.1 v2:172.21.15.80:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4090+0+0 (secure 0 0 0) 0x7f8c040852a0 con 0x7f8c08071250 2024-04-25T23:49:00.549 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.563+0000 7f8c0ca45640 1 --2- 172.21.15.57:0/188466202 >> v2:172.21.15.80:6816/2493457677 conn(0x7f8bec0677f0 0x7f8bec069cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.549 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.563+0000 7f8c0ca45640 1 --2- 172.21.15.57:0/188466202 >> v2:172.21.15.80:6816/2493457677 conn(0x7f8bec0677f0 0x7f8bec069cb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8c0813b180 tx=0x7f8bf8007a50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:00.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.563+0000 7f3c18974640 1 -- 172.21.15.57:0/1709096961 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c0c0a57e0 con 0x7f3c00008b50 2024-04-25T23:49:00.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.563+0000 7f8bfdffb640 1 -- 172.21.15.57:0/188466202 <== mon.1 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8c04056500 con 0x7f8c08071250 2024-04-25T23:49:00.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.563+0000 7f3c1bbda640 1 -- 172.21.15.57:0/1709096961 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c00008b50 msgr2=0x7f3c0000afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.563+0000 7f3c1bbda640 1 --2- 172.21.15.57:0/1709096961 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c00008b50 0x7f3c0000afe0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c0ab000 tx=0x7f3c0c009810 comp rx=0 tx=0).stop 2024-04-25T23:49:00.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.563+0000 7f3c1a177640 1 -- 172.21.15.57:0/1709096961 reap_dead start 2024-04-25T23:49:00.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.575+0000 7f3c1bbda640 1 -- 172.21.15.57:0/1709096961 shutdown_connections 2024-04-25T23:49:00.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.575+0000 7f3c1bbda640 1 --2- 172.21.15.57:0/1709096961 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c1406f390 0x7f3c14151490 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.575+0000 7f3c1bbda640 1 -- 172.21.15.57:0/1709096961 >> 172.21.15.57:0/1709096961 conn(0x7f3c140b4070 msgr2=0x7f3c140b4480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:49:00.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.575+0000 7f3c1bbda640 1 -- 172.21.15.57:0/1709096961 shutdown_connections 2024-04-25T23:49:00.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.575+0000 7f3c1bbda640 1 -- 172.21.15.57:0/1709096961 wait complete. 2024-04-25T23:49:00.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.575+0000 7f3c1bbda640 1 Processor -- start 2024-04-25T23:49:00.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.575+0000 7f3c1bbda640 1 -- start start 2024-04-25T23:49:00.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.575+0000 7f3c1bbda640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c1406f390 0x7f3c14144ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.575+0000 7f3c1bbda640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c00008b50 0x7f3c14145be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.575+0000 7f3c1bbda640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f3c140b2880 0x7f3c141df860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.575+0000 7f3c1bbda640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f3c14144350 con 0x7f3c1406f390 2024-04-25T23:49:00.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.575+0000 7f3c1bbda640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f3c141441d0 con 0x7f3c00008b50 2024-04-25T23:49:00.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.575+0000 7f3c1bbda640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f3c1414b460 con 0x7f3c140b2880 2024-04-25T23:49:00.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.575+0000 7f3c19976640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c1406f390 0x7f3c14144ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.575+0000 7f3c19976640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c1406f390 0x7f3c14144ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33558/0 (socket says 172.21.15.57:33558) 2024-04-25T23:49:00.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.575+0000 7f3c19976640 1 -- 172.21.15.57:0/741580239 learned_addr learned my addr 172.21.15.57:0/741580239 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:49:00.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.575+0000 7f3c19175640 1 --2- 172.21.15.57:0/741580239 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c00008b50 0x7f3c14145be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.575+0000 7f3c1a177640 1 --2- 172.21.15.57:0/741580239 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f3c140b2880 0x7f3c141df860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.575+0000 7f3c19175640 1 -- 172.21.15.57:0/741580239 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f3c140b2880 msgr2=0x7f3c141df860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.575+0000 7f3c19175640 1 --2- 172.21.15.57:0/741580239 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f3c140b2880 0x7f3c141df860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.575+0000 7f3c19175640 1 -- 172.21.15.57:0/741580239 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c1406f390 msgr2=0x7f3c14144ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.575+0000 7f3c19175640 1 --2- 172.21.15.57:0/741580239 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c1406f390 0x7f3c14144ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.579+0000 7f3c19175640 1 -- 172.21.15.57:0/741580239 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c14144980 con 0x7f3c00008b50 2024-04-25T23:49:00.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.579+0000 7f3c19976640 1 --2- 172.21.15.57:0/741580239 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c1406f390 0x7f3c14144ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T23:49:00.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.579+0000 7f3c19175640 1 --2- 172.21.15.57:0/741580239 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c00008b50 0x7f3c14145be0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3c0400c3f0 tx=0x7f3c0400c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:00.573 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.587+0000 7f3c0a7fc640 1 -- 172.21.15.57:0/741580239 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c04017070 con 0x7f3c00008b50 2024-04-25T23:49:00.573 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.587+0000 7f3c0a7fc640 1 -- 172.21.15.57:0/741580239 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c040052f0 con 0x7f3c00008b50 2024-04-25T23:49:00.573 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.587+0000 7f3c1bbda640 1 -- 172.21.15.57:0/741580239 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c14146180 con 0x7f3c00008b50 2024-04-25T23:49:00.573 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.587+0000 7f3c1bbda640 1 -- 172.21.15.57:0/741580239 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c141dff70 con 0x7f3c00008b50 2024-04-25T23:49:00.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.591+0000 7f3c0a7fc640 1 -- 172.21.15.57:0/741580239 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c04003840 con 0x7f3c00008b50 2024-04-25T23:49:00.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.591+0000 7f3c0a7fc640 1 -- 172.21.15.57:0/741580239 <== mon.1 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 4) ==== 87354+0+0 (secure 0 0 0) 0x7f3c0401a070 con 0x7f3c00008b50 2024-04-25T23:49:00.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.591+0000 7f3c0a7fc640 1 --2- 172.21.15.57:0/741580239 >> v2:172.21.15.80:6816/2493457677 conn(0x7f3c00069140 0x7f3c0006b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.578 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.591+0000 7f3c0a7fc640 1 -- 172.21.15.57:0/741580239 <== mon.1 v2:172.21.15.80:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4090+0+0 (secure 0 0 0) 0x7f3c04084830 con 0x7f3c00008b50 2024-04-25T23:49:00.578 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.591+0000 7f3c19976640 1 --2- 172.21.15.57:0/741580239 >> v2:172.21.15.80:6816/2493457677 conn(0x7f3c00069140 0x7f3c0006b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.578 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.591+0000 7f3c19976640 1 --2- 172.21.15.57:0/741580239 >> v2:172.21.15.80:6816/2493457677 conn(0x7f3c00069140 0x7f3c0006b600 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3c100093b0 tx=0x7f3c10009340 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:00.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.595+0000 7f3c1bbda640 1 -- 172.21.15.57:0/741580239 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bfc005090 con 0x7f3c00008b50 2024-04-25T23:49:00.583 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.595+0000 7fc4c59c1640 1 Processor -- start 2024-04-25T23:49:00.583 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.595+0000 7fc4c59c1640 1 -- start start 2024-04-25T23:49:00.584 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.595+0000 7fc4c59c1640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fc4c00b00f0 0x7fc4c0142f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.584 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.595+0000 7fc4c59c1640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4c00b15b0 con 0x7fc4c0074b50 2024-04-25T23:49:00.584 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.595+0000 7fc4c59c1640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4c0073420 con 0x7fc4c013feb0 2024-04-25T23:49:00.584 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.595+0000 7fc4c59c1640 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 -- 0x7fc4c01434d0 con 0x7fc4c00b00f0 2024-04-25T23:49:00.584 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.595+0000 7fc4bf7fe640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fc4c00b00f0 0x7fc4c0142f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.585 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.595+0000 7fc4be7fc640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fc4c013feb0 0x7fc4c0142880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.57:39682/0 (socket says 172.21.15.57:39682) 2024-04-25T23:49:00.585 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.595+0000 7fc4be7fc640 1 -- 172.21.15.57:0/2182278775 learned_addr learned my addr 172.21.15.57:0/2182278775 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-04-25T23:49:00.585 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.595+0000 7fc4bf7fe640 1 -- 172.21.15.57:0/2182278775 >> v1:172.21.15.80:6789/0 conn(0x7fc4c013feb0 legacy=0x7fc4c0142880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.585 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.595+0000 7fc4bf7fe640 1 -- 172.21.15.57:0/2182278775 >> v1:172.21.15.57:6789/0 conn(0x7fc4c0074b50 legacy=0x7fc4c013f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:49:00.585 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.595+0000 7fc4bf7fe640 1 -- 172.21.15.57:0/2182278775 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4c014fe80 con 0x7fc4c00b00f0 2024-04-25T23:49:00.588 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.599+0000 7fc4bf7fe640 1 --2- 172.21.15.57:0/2182278775 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4c00b00f0 0x7fc4c0142f90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc4b40097c0 tx=0x7fc4b401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acaec390d89a793c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:00.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.599+0000 7fc4bdffb640 1 -- 172.21.15.57:0/2182278775 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b40b0070 con 0x7fc4c00b00f0 2024-04-25T23:49:00.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.603+0000 7fc4bdffb640 1 -- 172.21.15.57:0/2182278775 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4b40ab600 con 0x7fc4c00b00f0 2024-04-25T23:49:00.589 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.603+0000 7fc4bdffb640 1 -- 172.21.15.57:0/2182278775 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b40aba10 con 0x7fc4c00b00f0 2024-04-25T23:49:00.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.603+0000 7fc4c59c1640 1 -- 172.21.15.57:0/2182278775 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4c00b00f0 msgr2=0x7fc4c0142f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.590 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.603+0000 7fc4c59c1640 1 --2- 172.21.15.57:0/2182278775 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4c00b00f0 0x7fc4c0142f90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc4b40097c0 tx=0x7fc4b401c460 comp rx=0 tx=0).stop 2024-04-25T23:49:00.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.603+0000 7fc4c59c1640 1 -- 172.21.15.57:0/2182278775 shutdown_connections 2024-04-25T23:49:00.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.603+0000 7fc4c59c1640 1 --2- 172.21.15.57:0/2182278775 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4c00b00f0 0x7fc4c0142f90 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.603+0000 7fc4c59c1640 1 -- 172.21.15.57:0/2182278775 >> 172.21.15.57:0/2182278775 conn(0x7fc4c00b4f30 msgr2=0x7fc4c00b5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:49:00.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.603+0000 7fc4c59c1640 1 -- 172.21.15.57:0/2182278775 shutdown_connections 2024-04-25T23:49:00.591 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.603+0000 7fc4c59c1640 1 -- 172.21.15.57:0/2182278775 wait complete. 2024-04-25T23:49:00.592 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.603+0000 7fc4c59c1640 1 Processor -- start 2024-04-25T23:49:00.593 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.603+0000 7fc4c59c1640 1 -- start start 2024-04-25T23:49:00.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.607+0000 7fc4c59c1640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc4c0074b50 0x7fc4c0145450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.607+0000 7fc4c59c1640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fc4c00b00f0 0x7fc4c01306c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.607+0000 7fc4c59c1640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4c013feb0 0x7fc4c0130c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.607+0000 7fc4c59c1640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c00b07b0 con 0x7fc4c013feb0 2024-04-25T23:49:00.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.607+0000 7fc4c59c1640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c0150a70 con 0x7fc4c0074b50 2024-04-25T23:49:00.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.607+0000 7fc4c59c1640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7fc4c0150bf0 con 0x7fc4c00b00f0 2024-04-25T23:49:00.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.607+0000 7fc4bf7fe640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4c013feb0 0x7fc4c0130c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.607+0000 7fc4bf7fe640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4c013feb0 0x7fc4c0130c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33570/0 (socket says 172.21.15.57:33570) 2024-04-25T23:49:00.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.607+0000 7fc4bf7fe640 1 -- 172.21.15.57:0/1222299572 learned_addr learned my addr 172.21.15.57:0/1222299572 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:49:00.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.607+0000 7fc4beffd640 1 --2- 172.21.15.57:0/1222299572 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc4c0074b50 0x7fc4c0145450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.607+0000 7fc4be7fc640 1 --2- 172.21.15.57:0/1222299572 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fc4c00b00f0 0x7fc4c01306c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.607+0000 7fc4bf7fe640 1 -- 172.21.15.57:0/1222299572 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4c013feb0 msgr2=0x7fc4c0130c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-04-25T23:49:00.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.607+0000 7fc4bf7fe640 1 -- 172.21.15.57:0/1222299572 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4c013feb0 msgr2=0x7fc4c0130c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T23:49:00.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.607+0000 7fc4bf7fe640 1 --2- 172.21.15.57:0/1222299572 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4c013feb0 0x7fc4c0130c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T23:49:00.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.607+0000 7fc4bf7fe640 1 --2- 172.21.15.57:0/1222299572 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4c013feb0 0x7fc4c0130c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T23:49:00.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.607+0000 7fc4be7fc640 1 -- 172.21.15.57:0/1222299572 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc4c0074b50 msgr2=0x7fc4c0145450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.607+0000 7fc4be7fc640 1 --2- 172.21.15.57:0/1222299572 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc4c0074b50 0x7fc4c0145450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.607+0000 7fc4be7fc640 1 -- 172.21.15.57:0/1222299572 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4c013feb0 msgr2=0x7fc4c0130c00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T23:49:00.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.607+0000 7fc4be7fc640 1 --2- 172.21.15.57:0/1222299572 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4c013feb0 0x7fc4c0130c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.607+0000 7fc4be7fc640 1 -- 172.21.15.57:0/1222299572 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4c00b4430 con 0x7fc4c00b00f0 2024-04-25T23:49:00.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.607+0000 7f3c0a7fc640 1 -- 172.21.15.57:0/741580239 <== mon.1 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3c040554c0 con 0x7f3c00008b50 2024-04-25T23:49:00.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.607+0000 7fc4be7fc640 1 --2- 172.21.15.57:0/1222299572 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fc4c00b00f0 0x7fc4c01306c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc4b000c370 tx=0x7fc4b000c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:00.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.675+0000 7fc4a7fff640 1 -- 172.21.15.57:0/1222299572 <== mon.2 v2:172.21.15.57:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b0017070 con 0x7fc4c00b00f0 2024-04-25T23:49:00.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.675+0000 7fc4c59c1640 1 -- 172.21.15.57:0/1222299572 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4c00b4710 con 0x7fc4c00b00f0 2024-04-25T23:49:00.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.675+0000 7fc4c59c1640 1 -- 172.21.15.57:0/1222299572 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4c00b2930 con 0x7fc4c00b00f0 2024-04-25T23:49:00.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.675+0000 7fc4a7fff640 1 -- 172.21.15.57:0/1222299572 <== mon.2 v2:172.21.15.57:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4b0005cf0 con 0x7fc4c00b00f0 2024-04-25T23:49:00.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.675+0000 7fc4a7fff640 1 -- 172.21.15.57:0/1222299572 <== mon.2 v2:172.21.15.57:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b00072d0 con 0x7fc4c00b00f0 2024-04-25T23:49:00.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.675+0000 7fc4c59c1640 1 -- 172.21.15.57:0/1222299572 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4a0005090 con 0x7fc4c00b00f0 2024-04-25T23:49:00.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.675+0000 7fc4a7fff640 1 -- 172.21.15.57:0/1222299572 <== mon.2 v2:172.21.15.57:3301/0 4 ==== mgrmap(e 4) ==== 87354+0+0 (secure 0 0 0) 0x7fc4b001a070 con 0x7fc4c00b00f0 2024-04-25T23:49:00.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.675+0000 7fc4a7fff640 1 --2- 172.21.15.57:0/1222299572 >> v2:172.21.15.80:6816/2493457677 conn(0x7fc4ac0677f0 0x7fc4ac069cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:00.665 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.675+0000 7fc4a7fff640 1 -- 172.21.15.57:0/1222299572 <== mon.2 v2:172.21.15.57:3301/0 5 ==== osd_map(17..17 src has 1..17) ==== 4090+0+0 (secure 0 0 0) 0x7fc4b0084680 con 0x7fc4c00b00f0 2024-04-25T23:49:00.665 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.679+0000 7fc4beffd640 1 --2- 172.21.15.57:0/1222299572 >> v2:172.21.15.80:6816/2493457677 conn(0x7fc4ac0677f0 0x7fc4ac069cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:00.675 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.687+0000 7fc4a7fff640 1 -- 172.21.15.57:0/1222299572 <== mon.2 v2:172.21.15.57:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc4b0055290 con 0x7fc4c00b00f0 2024-04-25T23:49:00.675 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.687+0000 7fc4beffd640 1 --2- 172.21.15.57:0/1222299572 >> v2:172.21.15.80:6816/2493457677 conn(0x7fc4ac0677f0 0x7fc4ac069cb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8005680 tx=0x7fc4b8003dd0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:00.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.787+0000 7f615359e640 1 -- 172.21.15.57:0/3862342756 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f6154147280 con 0x7f61541497d0 2024-04-25T23:49:00.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.787+0000 7f6142ffd640 1 -- 172.21.15.57:0/3862342756 <== mon.2 v2:172.21.15.57:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f614805a5b0 con 0x7f61541497d0 2024-04-25T23:49:00.778 INFO:teuthology.orchestra.run.smithi057.stdout:55834574849 2024-04-25T23:49:00.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.799+0000 7f6140ff9640 1 -- 172.21.15.57:0/3862342756 >> v2:172.21.15.80:6816/2493457677 conn(0x7f61340677f0 msgr2=0x7f6134069cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.799+0000 7f6140ff9640 1 --2- 172.21.15.57:0/3862342756 >> v2:172.21.15.80:6816/2493457677 conn(0x7f61340677f0 0x7f6134069cb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f61440af000 tx=0x7f61440a9000 comp rx=0 tx=0).stop 2024-04-25T23:49:00.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.799+0000 7f6140ff9640 1 -- 172.21.15.57:0/3862342756 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f61541497d0 msgr2=0x7f6154147590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.799+0000 7f6140ff9640 1 --2- 172.21.15.57:0/3862342756 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f61541497d0 0x7f6154147590 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f61540b1a20 tx=0x7f614800fd10 comp rx=0 tx=0).stop 2024-04-25T23:49:00.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.799+0000 7f6140ff9640 1 -- 172.21.15.57:0/3862342756 shutdown_connections 2024-04-25T23:49:00.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.799+0000 7f6140ff9640 1 --2- 172.21.15.57:0/3862342756 >> v2:172.21.15.80:6816/2493457677 conn(0x7f61340677f0 0x7f6134069cb0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.799+0000 7f6140ff9640 1 --2- 172.21.15.57:0/3862342756 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f61541424f0 0x7f6154147ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.799+0000 7f6140ff9640 1 --2- 172.21.15.57:0/3862342756 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f61541497d0 0x7f6154147590 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.799+0000 7f6140ff9640 1 --2- 172.21.15.57:0/3862342756 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f61541493f0 0x7f61541373c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.787 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.799+0000 7f6140ff9640 1 -- 172.21.15.57:0/3862342756 >> 172.21.15.57:0/3862342756 conn(0x7f61540b42a0 msgr2=0x7f61540b2360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:49:00.787 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.799+0000 7f6140ff9640 1 -- 172.21.15.57:0/3862342756 shutdown_connections 2024-04-25T23:49:00.787 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.799+0000 7f6140ff9640 1 -- 172.21.15.57:0/3862342756 wait complete. 2024-04-25T23:49:00.801 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.815+0000 7fa6f5795640 1 -- 172.21.15.57:0/403657498 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fa6d0005380 con 0x7fa6f006f0a0 2024-04-25T23:49:00.801 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.815+0000 7fa6ecff9640 1 -- 172.21.15.57:0/403657498 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa6e4056140 con 0x7fa6f006f0a0 2024-04-25T23:49:00.802 INFO:teuthology.orchestra.run.smithi057.stdout:55834574849 2024-04-25T23:49:00.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.815+0000 7fed4c783640 1 -- 172.21.15.57:0/2497365725 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fed2c005650 con 0x7fed440b53b0 2024-04-25T23:49:00.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.815+0000 7fed3affd640 1 -- 172.21.15.57:0/2497365725 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fed44148d80 con 0x7fed440b53b0 2024-04-25T23:49:00.803 INFO:teuthology.orchestra.run.smithi057.stdout:55834574849 2024-04-25T23:49:00.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.815+0000 7fa6f5795640 1 -- 172.21.15.57:0/403657498 >> v2:172.21.15.80:6816/2493457677 conn(0x7fa6d8067950 msgr2=0x7fa6d8069e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.815+0000 7fa6f5795640 1 --2- 172.21.15.57:0/403657498 >> v2:172.21.15.80:6816/2493457677 conn(0x7fa6d8067950 0x7fa6d8069e10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa6e80b2000 tx=0x7fa6e8004ac0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.815+0000 7fa6f5795640 1 -- 172.21.15.57:0/403657498 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa6f006f0a0 msgr2=0x7fa6f0141e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.815+0000 7fa6f5795640 1 --2- 172.21.15.57:0/403657498 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa6f006f0a0 0x7fa6f0141e60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa6e400c450 tx=0x7fa6e400c910 comp rx=0 tx=0).stop 2024-04-25T23:49:00.805 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.819+0000 7fa6f5795640 1 -- 172.21.15.57:0/403657498 shutdown_connections 2024-04-25T23:49:00.806 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.819+0000 7fa6f5795640 1 --2- 172.21.15.57:0/403657498 >> v2:172.21.15.80:6816/2493457677 conn(0x7fa6d8067950 0x7fa6d8069e10 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.806 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.819+0000 7fa6f5795640 1 --2- 172.21.15.57:0/403657498 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa6f013c2a0 0x7fa6f0131640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.806 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.819+0000 7fa6f5795640 1 --2- 172.21.15.57:0/403657498 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fa6f0071cd0 0x7fa6f0134980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.806 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.819+0000 7fa6f5795640 1 --2- 172.21.15.57:0/403657498 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa6f006f0a0 0x7fa6f0141e60 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.806 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.819+0000 7fa6f5795640 1 -- 172.21.15.57:0/403657498 >> 172.21.15.57:0/403657498 conn(0x7fa6f006b210 msgr2=0x7fa6f00b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:49:00.806 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.819+0000 7fa6f5795640 1 -- 172.21.15.57:0/403657498 shutdown_connections 2024-04-25T23:49:00.806 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.819+0000 7fa6f5795640 1 -- 172.21.15.57:0/403657498 wait complete. 2024-04-25T23:49:00.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.819+0000 7fed4c783640 1 -- 172.21.15.57:0/2497365725 >> v2:172.21.15.80:6816/2493457677 conn(0x7fed280678c0 msgr2=0x7fed28069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.819+0000 7fed4c783640 1 --2- 172.21.15.57:0/2497365725 >> v2:172.21.15.80:6816/2493457677 conn(0x7fed280678c0 0x7fed28069d80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fed44147480 tx=0x7fed40006c50 comp rx=0 tx=0).stop 2024-04-25T23:49:00.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.819+0000 7fed4c783640 1 -- 172.21.15.57:0/2497365725 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fed440b53b0 msgr2=0x7fed44146f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.819+0000 7fed4c783640 1 --2- 172.21.15.57:0/2497365725 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fed440b53b0 0x7fed44146f40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fed3401a030 tx=0x7fed340191a0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.819+0000 7fed4c783640 1 -- 172.21.15.57:0/2497365725 shutdown_connections 2024-04-25T23:49:00.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.819+0000 7fed4c783640 1 --2- 172.21.15.57:0/2497365725 >> v2:172.21.15.80:6816/2493457677 conn(0x7fed280678c0 0x7fed28069d80 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.819+0000 7fed4c783640 1 --2- 172.21.15.57:0/2497365725 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fed4413f850 0x7fed44140cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.819+0000 7fed4c783640 1 --2- 172.21.15.57:0/2497365725 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fed44145ed0 0x7fed44141200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.819+0000 7fed4c783640 1 --2- 172.21.15.57:0/2497365725 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fed440b53b0 0x7fed44146f40 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.819+0000 7fed4c783640 1 -- 172.21.15.57:0/2497365725 >> 172.21.15.57:0/2497365725 conn(0x7fed44074cf0 msgr2=0x7fed440750d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:49:00.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.819+0000 7fed4c783640 1 -- 172.21.15.57:0/2497365725 shutdown_connections 2024-04-25T23:49:00.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.819+0000 7fed4c783640 1 -- 172.21.15.57:0/2497365725 wait complete. 2024-04-25T23:49:00.809 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574849 for osd.0 2024-04-25T23:49:00.831 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574849 for osd.1 2024-04-25T23:49:00.832 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574849 for osd.2 2024-04-25T23:49:00.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.847+0000 7f9920ff9640 1 -- 172.21.15.57:0/503217135 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f992c0b3f40 con 0x7f992c06fa80 2024-04-25T23:49:00.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.847+0000 7f9922ffd640 1 -- 172.21.15.57:0/503217135 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f99240555a0 con 0x7f992c06fa80 2024-04-25T23:49:00.835 INFO:teuthology.orchestra.run.smithi057.stdout:55834574849 2024-04-25T23:49:00.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.851+0000 7f9920ff9640 1 -- 172.21.15.57:0/503217135 >> v2:172.21.15.80:6816/2493457677 conn(0x7f99100677f0 msgr2=0x7f9910069cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.851+0000 7f9920ff9640 1 --2- 172.21.15.57:0/503217135 >> v2:172.21.15.80:6816/2493457677 conn(0x7f99100677f0 0x7f9910069cb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f991c0ae000 tx=0x7f991c0a7070 comp rx=0 tx=0).stop 2024-04-25T23:49:00.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.851+0000 7f9920ff9640 1 -- 172.21.15.57:0/503217135 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f992c06fa80 msgr2=0x7f992c1eaa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.851+0000 7f9920ff9640 1 --2- 172.21.15.57:0/503217135 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f992c06fa80 0x7f992c1eaa00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f992400c710 tx=0x7f992400cbd0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.851+0000 7f9920ff9640 1 -- 172.21.15.57:0/503217135 shutdown_connections 2024-04-25T23:49:00.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.851+0000 7f9920ff9640 1 --2- 172.21.15.57:0/503217135 >> v2:172.21.15.80:6816/2493457677 conn(0x7f99100677f0 0x7f9910069cb0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.851+0000 7f9920ff9640 1 --2- 172.21.15.57:0/503217135 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f992c06edf0 0x7f992c14c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.851+0000 7f9920ff9640 1 --2- 172.21.15.57:0/503217135 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f992c06fa80 0x7f992c1eaa00 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.851+0000 7f9920ff9640 1 --2- 172.21.15.57:0/503217135 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f992c06ea10 0x7f992c0a9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.851+0000 7f9920ff9640 1 -- 172.21.15.57:0/503217135 >> 172.21.15.57:0/503217135 conn(0x7f992c0752a0 msgr2=0x7f992c089f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:49:00.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.851+0000 7f9920ff9640 1 -- 172.21.15.57:0/503217135 shutdown_connections 2024-04-25T23:49:00.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.851+0000 7f9920ff9640 1 -- 172.21.15.57:0/503217135 wait complete. 2024-04-25T23:49:00.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.855+0000 7f67d109e640 1 -- 172.21.15.57:0/839849140 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f67ac005650 con 0x7f67cc142aa0 2024-04-25T23:49:00.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.855+0000 7f67c8ff9640 1 -- 172.21.15.57:0/839849140 <== mon.2 v2:172.21.15.57:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f67c40187c0 con 0x7f67cc142aa0 2024-04-25T23:49:00.842 INFO:teuthology.orchestra.run.smithi057.stdout:60129542145 2024-04-25T23:49:00.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.859+0000 7f67d109e640 1 -- 172.21.15.57:0/839849140 >> v2:172.21.15.80:6816/2493457677 conn(0x7f67b80677f0 msgr2=0x7f67b8069cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.846 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.859+0000 7f67d109e640 1 --2- 172.21.15.57:0/839849140 >> v2:172.21.15.80:6816/2493457677 conn(0x7f67b80677f0 0x7f67b8069cb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f67bc009730 tx=0x7f67bc006ca0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.846 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.859+0000 7f67d109e640 1 -- 172.21.15.57:0/839849140 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f67cc142aa0 msgr2=0x7f67cc13dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.846 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.859+0000 7f67d109e640 1 --2- 172.21.15.57:0/839849140 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f67cc142aa0 0x7f67cc13dc50 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f67c401d160 tx=0x7f67c401b850 comp rx=0 tx=0).stop 2024-04-25T23:49:00.846 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.859+0000 7f67d109e640 1 -- 172.21.15.57:0/839849140 shutdown_connections 2024-04-25T23:49:00.846 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.859+0000 7f67d109e640 1 --2- 172.21.15.57:0/839849140 >> v2:172.21.15.80:6816/2493457677 conn(0x7f67b80677f0 0x7f67b8069cb0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.846 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.859+0000 7f67d109e640 1 --2- 172.21.15.57:0/839849140 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67cc070b70 0x7f67cc13c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.846 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.859+0000 7f67d109e640 1 --2- 172.21.15.57:0/839849140 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f67cc142aa0 0x7f67cc13dc50 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.846 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.859+0000 7f67d109e640 1 --2- 172.21.15.57:0/839849140 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f67cc070790 0x7f67cc1372d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.846 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.859+0000 7f67d109e640 1 -- 172.21.15.57:0/839849140 >> 172.21.15.57:0/839849140 conn(0x7f67cc06b3e0 msgr2=0x7f67cc0725c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:49:00.846 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.859+0000 7f67d109e640 1 -- 172.21.15.57:0/839849140 shutdown_connections 2024-04-25T23:49:00.846 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.859+0000 7f67d109e640 1 -- 172.21.15.57:0/839849140 wait complete. 2024-04-25T23:49:00.862 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574849 for osd.3 2024-04-25T23:49:00.868 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542146 got 60129542145 for osd.4 2024-04-25T23:49:00.874 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.887+0000 7f8c0e248640 1 -- 172.21.15.57:0/188466202 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f8c0813e420 con 0x7f8c08071250 2024-04-25T23:49:00.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.887+0000 7f8bfdffb640 1 -- 172.21.15.57:0/188466202 <== mon.1 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8c04055e00 con 0x7f8c08071250 2024-04-25T23:49:00.875 INFO:teuthology.orchestra.run.smithi057.stdout:60129542146 2024-04-25T23:49:00.877 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.891+0000 7f8c0e248640 1 -- 172.21.15.57:0/188466202 >> v2:172.21.15.80:6816/2493457677 conn(0x7f8bec0677f0 msgr2=0x7f8bec069cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.891+0000 7f8c0e248640 1 --2- 172.21.15.57:0/188466202 >> v2:172.21.15.80:6816/2493457677 conn(0x7f8bec0677f0 0x7f8bec069cb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8c0813b180 tx=0x7f8bf8007a50 comp rx=0 tx=0).stop 2024-04-25T23:49:00.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.891+0000 7f8c0e248640 1 -- 172.21.15.57:0/188466202 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8c08071250 msgr2=0x7f8c0813ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.891+0000 7f8c0e248640 1 --2- 172.21.15.57:0/188466202 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8c08071250 0x7f8c0813ac40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8c0400c500 tx=0x7f8c0400c9c0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.891+0000 7f8c0e248640 1 -- 172.21.15.57:0/188466202 shutdown_connections 2024-04-25T23:49:00.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.891+0000 7f8c0e248640 1 --2- 172.21.15.57:0/188466202 >> v2:172.21.15.80:6816/2493457677 conn(0x7f8bec0677f0 0x7f8bec069cb0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.891+0000 7f8c0e248640 1 --2- 172.21.15.57:0/188466202 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f8c0813a370 0x7f8c0812f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.891+0000 7f8c0e248640 1 --2- 172.21.15.57:0/188466202 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c080724c0 0x7f8c081dbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.891+0000 7f8c0e248640 1 --2- 172.21.15.57:0/188466202 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8c08071250 0x7f8c0813ac40 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.891+0000 7f8c0e248640 1 -- 172.21.15.57:0/188466202 >> 172.21.15.57:0/188466202 conn(0x7f8c080b62c0 msgr2=0x7f8c080b56b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:49:00.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.891+0000 7f8c0e248640 1 -- 172.21.15.57:0/188466202 shutdown_connections 2024-04-25T23:49:00.878 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.891+0000 7f8c0e248640 1 -- 172.21.15.57:0/188466202 wait complete. 2024-04-25T23:49:00.892 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542147 got 60129542146 for osd.5 2024-04-25T23:49:00.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.911+0000 7f3c1bbda640 1 -- 172.21.15.57:0/741580239 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f3bfc005380 con 0x7f3c00008b50 2024-04-25T23:49:00.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.911+0000 7f3c0a7fc640 1 -- 172.21.15.57:0/741580239 <== mon.1 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3c040597c0 con 0x7f3c00008b50 2024-04-25T23:49:00.898 INFO:teuthology.orchestra.run.smithi057.stdout:60129542145 2024-04-25T23:49:00.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.911+0000 7f3c1bbda640 1 -- 172.21.15.57:0/741580239 >> v2:172.21.15.80:6816/2493457677 conn(0x7f3c00069140 msgr2=0x7f3c0006b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.911+0000 7f3c1bbda640 1 --2- 172.21.15.57:0/741580239 >> v2:172.21.15.80:6816/2493457677 conn(0x7f3c00069140 0x7f3c0006b600 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3c100093b0 tx=0x7f3c10009340 comp rx=0 tx=0).stop 2024-04-25T23:49:00.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.911+0000 7f3c1bbda640 1 -- 172.21.15.57:0/741580239 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c00008b50 msgr2=0x7f3c14145be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.911+0000 7f3c1bbda640 1 --2- 172.21.15.57:0/741580239 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c00008b50 0x7f3c14145be0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3c0400c3f0 tx=0x7f3c0400c8b0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.911+0000 7f3c1bbda640 1 -- 172.21.15.57:0/741580239 shutdown_connections 2024-04-25T23:49:00.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.911+0000 7f3c1bbda640 1 --2- 172.21.15.57:0/741580239 >> v2:172.21.15.80:6816/2493457677 conn(0x7f3c00069140 0x7f3c0006b600 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.911+0000 7f3c1bbda640 1 --2- 172.21.15.57:0/741580239 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c00008b50 0x7f3c14145be0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.911+0000 7f3c1bbda640 1 --2- 172.21.15.57:0/741580239 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f3c140b2880 0x7f3c141df860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.911+0000 7f3c1bbda640 1 --2- 172.21.15.57:0/741580239 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c1406f390 0x7f3c14144ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.911+0000 7f3c1bbda640 1 -- 172.21.15.57:0/741580239 >> 172.21.15.57:0/741580239 conn(0x7f3c140b4070 msgr2=0x7f3c140b1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:49:00.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.911+0000 7f3c1bbda640 1 -- 172.21.15.57:0/741580239 shutdown_connections 2024-04-25T23:49:00.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.911+0000 7f3c1bbda640 1 -- 172.21.15.57:0/741580239 wait complete. 2024-04-25T23:49:00.901 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.915+0000 7fc4c59c1640 1 -- 172.21.15.57:0/1222299572 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fc4a0005380 con 0x7fc4c00b00f0 2024-04-25T23:49:00.910 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.923+0000 7fc4a7fff640 1 -- 172.21.15.57:0/1222299572 <== mon.2 v2:172.21.15.57:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc4b0059590 con 0x7fc4c00b00f0 2024-04-25T23:49:00.910 INFO:teuthology.orchestra.run.smithi057.stdout:55834574849 2024-04-25T23:49:00.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.923+0000 7fc4c59c1640 1 -- 172.21.15.57:0/1222299572 >> v2:172.21.15.80:6816/2493457677 conn(0x7fc4ac0677f0 msgr2=0x7fc4ac069cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.923+0000 7fc4c59c1640 1 --2- 172.21.15.57:0/1222299572 >> v2:172.21.15.80:6816/2493457677 conn(0x7fc4ac0677f0 0x7fc4ac069cb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8005680 tx=0x7fc4b8003dd0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.923+0000 7fc4c59c1640 1 -- 172.21.15.57:0/1222299572 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fc4c00b00f0 msgr2=0x7fc4c01306c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:00.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.923+0000 7fc4c59c1640 1 --2- 172.21.15.57:0/1222299572 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fc4c00b00f0 0x7fc4c01306c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc4b000c370 tx=0x7fc4b000c830 comp rx=0 tx=0).stop 2024-04-25T23:49:00.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.923+0000 7fc4c59c1640 1 -- 172.21.15.57:0/1222299572 shutdown_connections 2024-04-25T23:49:00.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.923+0000 7fc4c59c1640 1 --2- 172.21.15.57:0/1222299572 >> v2:172.21.15.80:6816/2493457677 conn(0x7fc4ac0677f0 0x7fc4ac069cb0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.923+0000 7fc4c59c1640 1 --2- 172.21.15.57:0/1222299572 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4c013feb0 0x7fc4c0130c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.923+0000 7fc4c59c1640 1 --2- 172.21.15.57:0/1222299572 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fc4c00b00f0 0x7fc4c01306c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.923+0000 7fc4c59c1640 1 --2- 172.21.15.57:0/1222299572 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc4c0074b50 0x7fc4c0145450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:00.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.923+0000 7fc4c59c1640 1 -- 172.21.15.57:0/1222299572 >> 172.21.15.57:0/1222299572 conn(0x7fc4c00b4f30 msgr2=0x7fc4c01346e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:49:00.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.923+0000 7fc4c59c1640 1 -- 172.21.15.57:0/1222299572 shutdown_connections 2024-04-25T23:49:00.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:00.923+0000 7fc4c59c1640 1 -- 172.21.15.57:0/1222299572 wait complete. 2024-04-25T23:49:00.917 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542146 got 60129542145 for osd.6 2024-04-25T23:49:00.931 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574849 for osd.7 2024-04-25T23:49:01.810 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T23:49:01.832 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T23:49:01.832 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T23:49:01.863 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T23:49:01.869 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T23:49:01.892 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T23:49:01.918 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T23:49:01.932 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T23:49:01.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:01.983+0000 7fcb4b5ff640 1 Processor -- start 2024-04-25T23:49:01.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:01.983+0000 7fcb4b5ff640 1 -- start start 2024-04-25T23:49:01.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:01.983+0000 7fcb4b5ff640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fcb4406f1a0 0x7fcb4413bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:01.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:01.983+0000 7fcb4b5ff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fcb4413c1c0 0x7fcb44072790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:01.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:01.983+0000 7fcb4b5ff640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb440713f0 con 0x7fcb4406f580 2024-04-25T23:49:01.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:01.983+0000 7fcb4b5ff640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fcb44141cb0 con 0x7fcb4413c1c0 2024-04-25T23:49:01.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:01.983+0000 7fcb4b5ff640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7fcb44070530 con 0x7fcb4406f1a0 2024-04-25T23:49:01.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:01.983+0000 7fcb4939b640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fcb4406f1a0 0x7fcb4413bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:01.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:01.983+0000 7fcb4939b640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fcb4406f1a0 0x7fcb4413bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3301/0 says I am v2:172.21.15.57:47478/0 (socket says 172.21.15.57:47478) 2024-04-25T23:49:01.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:01.983+0000 7fcb4939b640 1 -- 172.21.15.57:0/3552958267 learned_addr learned my addr 172.21.15.57:0/3552958267 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:49:01.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:01.983+0000 7fcb49b9c640 1 --2- 172.21.15.57:0/3552958267 >> v2:172.21.15.80:3300/0 conn(0x7fcb4413c1c0 0x7fcb44072790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:01.972 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:01.983+0000 7fcb4939b640 1 -- 172.21.15.57:0/3552958267 >> v2:172.21.15.80:3300/0 conn(0x7fcb4413c1c0 msgr2=0x7fcb44072790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:49:01.972 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:01.983+0000 7fcb4939b640 1 --2- 172.21.15.57:0/3552958267 >> v2:172.21.15.80:3300/0 conn(0x7fcb4413c1c0 0x7fcb44072790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:01.972 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:01.983+0000 7fcb4939b640 1 -- 172.21.15.57:0/3552958267 >> v1:172.21.15.80:6789/0 conn(0x7fcb4406f580 legacy=0x7fcb44072080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:01.972 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:01.983+0000 7fcb4939b640 1 -- 172.21.15.57:0/3552958267 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb440716d0 con 0x7fcb4406f1a0 2024-04-25T23:49:01.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:01.987+0000 7fcb49b9c640 1 --2- 172.21.15.57:0/3552958267 >> v2:172.21.15.80:3300/0 conn(0x7fcb4413c1c0 0x7fcb44072790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T23:49:01.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:01.987+0000 7fcb4939b640 1 --2- 172.21.15.57:0/3552958267 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fcb4406f1a0 0x7fcb4413bc80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcb40009870 tx=0x7fcb4001c360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2b976ee917c378d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:01.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:01.987+0000 7fcb3bfff640 1 -- 172.21.15.57:0/3552958267 <== mon.2 v2:172.21.15.57:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb400b0070 con 0x7fcb4406f1a0 2024-04-25T23:49:01.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:01.987+0000 7fcb3bfff640 1 -- 172.21.15.57:0/3552958267 <== mon.2 v2:172.21.15.57:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb400ab600 con 0x7fcb4406f1a0 2024-04-25T23:49:01.977 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:01.991+0000 7fcb4b5ff640 1 -- 172.21.15.57:0/3552958267 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fcb4406f1a0 msgr2=0x7fcb4413bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:01.977 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:01.991+0000 7fcb4b5ff640 1 --2- 172.21.15.57:0/3552958267 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fcb4406f1a0 0x7fcb4413bc80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcb40009870 tx=0x7fcb4001c360 comp rx=0 tx=0).stop 2024-04-25T23:49:01.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:01.995+0000 7fcb4b5ff640 1 -- 172.21.15.57:0/3552958267 shutdown_connections 2024-04-25T23:49:01.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:01.995+0000 7fcb4b5ff640 1 --2- 172.21.15.57:0/3552958267 >> v2:172.21.15.80:3300/0 conn(0x7fcb4413c1c0 0x7fcb44072790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:01.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:01.995+0000 7fcb4b5ff640 1 --2- 172.21.15.57:0/3552958267 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fcb4406f1a0 0x7fcb4413bc80 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:01.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:01.995+0000 7fcb4b5ff640 1 -- 172.21.15.57:0/3552958267 >> 172.21.15.57:0/3552958267 conn(0x7fcb440b4070 msgr2=0x7fcb440b4480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:49:01.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:01.995+0000 7fcb4b5ff640 1 -- 172.21.15.57:0/3552958267 shutdown_connections 2024-04-25T23:49:01.983 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:01.995+0000 7fcb4b5ff640 1 -- 172.21.15.57:0/3552958267 wait complete. 2024-04-25T23:49:01.983 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:01.995+0000 7fcb4b5ff640 1 Processor -- start 2024-04-25T23:49:01.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:01.995+0000 7fcb4b5ff640 1 -- start start 2024-04-25T23:49:01.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:01.995+0000 7fcb4b5ff640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fcb4406f1a0 0x7fcb441349f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:01.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:01.995+0000 7fcb4b5ff640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb4412d320 0x7fcb4413e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:01.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:01.995+0000 7fcb4b5ff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcb4412d780 0x7fcb4413ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:01.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:01.995+0000 7fcb4b5ff640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4414c0f0 con 0x7fcb4412d320 2024-04-25T23:49:01.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:01.995+0000 7fcb4b5ff640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fcb440aeaf0 con 0x7fcb4412d780 2024-04-25T23:49:01.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:01.995+0000 7fcb4b5ff640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7fcb4414c270 con 0x7fcb4406f1a0 2024-04-25T23:49:01.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:01.995+0000 7fcb48b9a640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb4412d320 0x7fcb4413e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:01.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:01.995+0000 7fcb4939b640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fcb4406f1a0 0x7fcb441349f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:01.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:01.995+0000 7fcb48b9a640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb4412d320 0x7fcb4413e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33576/0 (socket says 172.21.15.57:33576) 2024-04-25T23:49:01.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:01.995+0000 7fcb48b9a640 1 -- 172.21.15.57:0/128034274 learned_addr learned my addr 172.21.15.57:0/128034274 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:49:01.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:01.999+0000 7fcb4939b640 1 -- 172.21.15.57:0/128034274 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcb4412d780 msgr2=0x7fcb4413ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:01.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:01.999+0000 7fcb4939b640 1 --2- 172.21.15.57:0/128034274 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcb4412d780 0x7fcb4413ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:01.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:01.999+0000 7fcb4939b640 1 -- 172.21.15.57:0/128034274 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb4412d320 msgr2=0x7fcb4413e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:01.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:01.999+0000 7fcb4939b640 1 --2- 172.21.15.57:0/128034274 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb4412d320 0x7fcb4413e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:01.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:01.999+0000 7fcb4939b640 1 -- 172.21.15.57:0/128034274 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb441ded50 con 0x7fcb4406f1a0 2024-04-25T23:49:01.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:01.999+0000 7fcb48b9a640 1 --2- 172.21.15.57:0/128034274 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb4412d320 0x7fcb4413e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T23:49:01.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:01.999+0000 7fcb4939b640 1 --2- 172.21.15.57:0/128034274 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fcb4406f1a0 0x7fcb441349f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fcb440706b0 tx=0x7fcb40019100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:01.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:01.999+0000 7fcb39ffb640 1 -- 172.21.15.57:0/128034274 <== mon.2 v2:172.21.15.57:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb400b9020 con 0x7fcb4406f1a0 2024-04-25T23:49:01.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:01.999+0000 7fcb4b5ff640 1 -- 172.21.15.57:0/128034274 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb44134f30 con 0x7fcb4406f1a0 2024-04-25T23:49:01.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:01.999+0000 7fcb4b5ff640 1 -- 172.21.15.57:0/128034274 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb4413c9c0 con 0x7fcb4406f1a0 2024-04-25T23:49:01.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.003+0000 7fcb39ffb640 1 -- 172.21.15.57:0/128034274 <== mon.2 v2:172.21.15.57:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb400aa970 con 0x7fcb4406f1a0 2024-04-25T23:49:01.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.003+0000 7fcb39ffb640 1 -- 172.21.15.57:0/128034274 <== mon.2 v2:172.21.15.57:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb400b0040 con 0x7fcb4406f1a0 2024-04-25T23:49:01.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.003+0000 7fcb39ffb640 1 -- 172.21.15.57:0/128034274 <== mon.2 v2:172.21.15.57:3301/0 4 ==== mgrmap(e 5) ==== 87410+0+0 (secure 0 0 0) 0x7fcb400bc020 con 0x7fcb4406f1a0 2024-04-25T23:49:01.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.003+0000 7fcb39ffb640 1 --2- 172.21.15.57:0/128034274 >> v2:172.21.15.80:6816/2493457677 conn(0x7fcb30067af0 0x7fcb30069fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:01.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.003+0000 7fcb39ffb640 1 -- 172.21.15.57:0/128034274 <== mon.2 v2:172.21.15.57:3301/0 5 ==== osd_map(17..17 src has 1..17) ==== 4090+0+0 (secure 0 0 0) 0x7fcb4011f840 con 0x7fcb4406f1a0 2024-04-25T23:49:01.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.003+0000 7fcb48b9a640 1 --2- 172.21.15.57:0/128034274 >> v2:172.21.15.80:6816/2493457677 conn(0x7fcb30067af0 0x7fcb30069fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:01.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.007+0000 7fcb48b9a640 1 --2- 172.21.15.57:0/128034274 >> v2:172.21.15.80:6816/2493457677 conn(0x7fcb30067af0 0x7fcb30069fb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c000f90 tx=0x7fcb3c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:01.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.011+0000 7fcb4b5ff640 1 -- 172.21.15.57:0/128034274 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb20005090 con 0x7fcb4406f1a0 2024-04-25T23:49:02.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.019+0000 7fcb39ffb640 1 -- 172.21.15.57:0/128034274 <== mon.2 v2:172.21.15.57:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fcb400c3030 con 0x7fcb4406f1a0 2024-04-25T23:49:02.013 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.023+0000 7f265ea43640 1 Processor -- start 2024-04-25T23:49:02.013 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.023+0000 7f265ea43640 1 -- start start 2024-04-25T23:49:02.013 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.023+0000 7f265ea43640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f26580700d0 0x7f2658152fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:02.013 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.023+0000 7f265ea43640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2658070fa0 con 0x7f265806efb0 2024-04-25T23:49:02.013 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.023+0000 7f265ea43640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2658071280 con 0x7f2658153500 2024-04-25T23:49:02.013 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.023+0000 7f265ea43640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f265806f390 con 0x7f26580700d0 2024-04-25T23:49:02.013 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.023+0000 7f2657fff640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f265806efb0 0x7f26580ae8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:49682/0 (socket says 172.21.15.57:49682) 2024-04-25T23:49:02.013 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.023+0000 7f2657fff640 1 -- 172.21.15.57:0/1031013996 learned_addr learned my addr 172.21.15.57:0/1031013996 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-04-25T23:49:02.013 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.023+0000 7f26577fe640 1 --2- 172.21.15.57:0/1031013996 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f26580700d0 0x7f2658152fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:02.013 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.023+0000 7f2656ffd640 1 -- 172.21.15.57:0/1031013996 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2883718008 0 0) 0x7f2658070fa0 con 0x7f265806efb0 2024-04-25T23:49:02.014 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.023+0000 7f26577fe640 1 -- 172.21.15.57:0/1031013996 >> v1:172.21.15.80:6789/0 conn(0x7f2658153500 legacy=0x7f2658159e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:02.014 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.023+0000 7f26577fe640 1 -- 172.21.15.57:0/1031013996 >> v1:172.21.15.57:6789/0 conn(0x7f265806efb0 legacy=0x7f26580ae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:02.014 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.023+0000 7f26577fe640 1 -- 172.21.15.57:0/1031013996 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2658071560 con 0x7f26580700d0 2024-04-25T23:49:02.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.027+0000 7f26577fe640 1 --2- 172.21.15.57:0/1031013996 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f26580700d0 0x7f2658152fc0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2648007070 tx=0x7f26480205d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61e50404fd43651b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:02.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.027+0000 7f2656ffd640 1 -- 172.21.15.57:0/1031013996 <== mon.2 v2:172.21.15.57:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2648007b90 con 0x7f26580700d0 2024-04-25T23:49:02.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.027+0000 7f2656ffd640 1 -- 172.21.15.57:0/1031013996 <== mon.2 v2:172.21.15.57:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26480b1070 con 0x7f26580700d0 2024-04-25T23:49:02.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.027+0000 7f2656ffd640 1 -- 172.21.15.57:0/1031013996 <== mon.2 v2:172.21.15.57:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26480090d0 con 0x7f26580700d0 2024-04-25T23:49:02.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.031+0000 7f265ea43640 1 -- 172.21.15.57:0/1031013996 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f26580700d0 msgr2=0x7f2658152fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:02.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.031+0000 7f265ea43640 1 --2- 172.21.15.57:0/1031013996 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f26580700d0 0x7f2658152fc0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2648007070 tx=0x7f26480205d0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.031+0000 7f265ea43640 1 -- 172.21.15.57:0/1031013996 shutdown_connections 2024-04-25T23:49:02.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.031+0000 7f265ea43640 1 --2- 172.21.15.57:0/1031013996 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f26580700d0 0x7f2658152fc0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.031+0000 7f265ea43640 1 -- 172.21.15.57:0/1031013996 >> 172.21.15.57:0/1031013996 conn(0x7f26580b4070 msgr2=0x7f26580b4480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:49:02.024 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.031+0000 7f265ea43640 1 -- 172.21.15.57:0/1031013996 shutdown_connections 2024-04-25T23:49:02.024 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.031+0000 7f265ea43640 1 -- 172.21.15.57:0/1031013996 wait complete. 2024-04-25T23:49:02.024 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.031+0000 7f265ea43640 1 Processor -- start 2024-04-25T23:49:02.024 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.035+0000 7f265ea43640 1 -- start start 2024-04-25T23:49:02.024 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.035+0000 7f265ea43640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f265806efb0 0x7f2658154030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:02.024 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.035+0000 7f265ea43640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f26580700d0 0x7f2658154d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:02.024 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.035+0000 7f265ea43640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2658153500 0x7f26581568c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:02.024 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.035+0000 7f265ea43640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f2658072b10 con 0x7f2658153500 2024-04-25T23:49:02.024 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.035+0000 7f265ea43640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f2658072990 con 0x7f265806efb0 2024-04-25T23:49:02.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.035+0000 7f265ea43640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f265815dd70 con 0x7f26580700d0 2024-04-25T23:49:02.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.035+0000 7f26577fe640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f26580700d0 0x7f2658154d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:02.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.035+0000 7f26577fe640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f26580700d0 0x7f2658154d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3301/0 says I am v2:172.21.15.57:47508/0 (socket says 172.21.15.57:47508) 2024-04-25T23:49:02.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.035+0000 7f26577fe640 1 -- 172.21.15.57:0/169899207 learned_addr learned my addr 172.21.15.57:0/169899207 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:49:02.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.035+0000 7f265cfe0640 1 --2- 172.21.15.57:0/169899207 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2658153500 0x7f26581568c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:02.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.035+0000 7f2657fff640 1 --2- 172.21.15.57:0/169899207 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f265806efb0 0x7f2658154030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:02.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.035+0000 7f26577fe640 1 -- 172.21.15.57:0/169899207 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f265806efb0 msgr2=0x7f2658154030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:02.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.035+0000 7f26577fe640 1 --2- 172.21.15.57:0/169899207 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f265806efb0 0x7f2658154030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.035+0000 7f26577fe640 1 -- 172.21.15.57:0/169899207 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2658153500 msgr2=0x7f26581568c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:02.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.035+0000 7f26577fe640 1 --2- 172.21.15.57:0/169899207 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2658153500 0x7f26581568c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.025 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.035+0000 7f26577fe640 1 -- 172.21.15.57:0/169899207 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2658154570 con 0x7f26580700d0 2024-04-25T23:49:02.026 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.035+0000 7f265cfe0640 1 --2- 172.21.15.57:0/169899207 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2658153500 0x7f26581568c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T23:49:02.026 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.035+0000 7f26577fe640 1 --2- 172.21.15.57:0/169899207 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f26580700d0 0x7f2658154d10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f26480ae040 tx=0x7f2648009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:02.026 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.035+0000 7f2654ff9640 1 -- 172.21.15.57:0/169899207 <== mon.2 v2:172.21.15.57:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26480bb070 con 0x7f26580700d0 2024-04-25T23:49:02.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.035+0000 7f2654ff9640 1 -- 172.21.15.57:0/169899207 <== mon.2 v2:172.21.15.57:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2648009720 con 0x7f26580700d0 2024-04-25T23:49:02.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.035+0000 7f265ea43640 1 -- 172.21.15.57:0/169899207 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2658155250 con 0x7f26580700d0 2024-04-25T23:49:02.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.035+0000 7f265ea43640 1 -- 172.21.15.57:0/169899207 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f26581564a0 con 0x7f26580700d0 2024-04-25T23:49:02.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.039+0000 7f2654ff9640 1 -- 172.21.15.57:0/169899207 <== mon.2 v2:172.21.15.57:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2648004220 con 0x7f26580700d0 2024-04-25T23:49:02.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.039+0000 7f2654ff9640 1 -- 172.21.15.57:0/169899207 <== mon.2 v2:172.21.15.57:3301/0 4 ==== mgrmap(e 5) ==== 87410+0+0 (secure 0 0 0) 0x7f26480120b0 con 0x7f26580700d0 2024-04-25T23:49:02.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.039+0000 7f2654ff9640 1 --2- 172.21.15.57:0/169899207 >> v2:172.21.15.80:6816/2493457677 conn(0x7f263c0679b0 0x7f263c069e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:02.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.039+0000 7f2654ff9640 1 -- 172.21.15.57:0/169899207 <== mon.2 v2:172.21.15.57:3301/0 5 ==== osd_map(17..17 src has 1..17) ==== 4090+0+0 (secure 0 0 0) 0x7f2648121830 con 0x7f26580700d0 2024-04-25T23:49:02.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.039+0000 7f2657fff640 1 --2- 172.21.15.57:0/169899207 >> v2:172.21.15.80:6816/2493457677 conn(0x7f263c0679b0 0x7f263c069e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:02.029 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.043+0000 7f2657fff640 1 --2- 172.21.15.57:0/169899207 >> v2:172.21.15.80:6816/2493457677 conn(0x7f263c0679b0 0x7f263c069e70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f26500012f0 tx=0x7f2650001770 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:02.029 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.043+0000 7f265ea43640 1 -- 172.21.15.57:0/169899207 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2634005090 con 0x7f26580700d0 2024-04-25T23:49:02.035 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.047+0000 7f7dc0989640 1 Processor -- start 2024-04-25T23:49:02.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.047+0000 7f7dc0989640 1 -- start start 2024-04-25T23:49:02.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.051+0000 7f7dc0989640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f7dbc074b50 0x7f7dbc0afff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:02.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.051+0000 7f7dc0989640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f7dbc14a2c0 0x7f7dbc14a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:02.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.051+0000 7f7dc0989640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7dbc0b1640 con 0x7f7dbc0b0530 2024-04-25T23:49:02.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.051+0000 7f7dc0989640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f7dbc14ac50 con 0x7f7dbc074b50 2024-04-25T23:49:02.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.051+0000 7f7dc0989640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f7dbc074f30 con 0x7f7dbc14a2c0 2024-04-25T23:49:02.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.051+0000 7f7dbbfff640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f7dbc14a2c0 0x7f7dbc14a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:02.038 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.051+0000 7f7dbaffd640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f7dbc0b0530 0x7f7dbc06fb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.57:39716/0 (socket says 172.21.15.57:39716) 2024-04-25T23:49:02.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.051+0000 7f7dbaffd640 1 -- 172.21.15.57:0/1059820382 learned_addr learned my addr 172.21.15.57:0/1059820382 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-04-25T23:49:02.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.051+0000 7f7dbb7fe640 1 --2- 172.21.15.57:0/1059820382 >> v2:172.21.15.80:3300/0 conn(0x7f7dbc074b50 0x7f7dbc0afff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:02.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.051+0000 7f7dbbfff640 1 -- 172.21.15.57:0/1059820382 >> v2:172.21.15.80:3300/0 conn(0x7f7dbc074b50 msgr2=0x7f7dbc0afff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:49:02.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.051+0000 7f7dbbfff640 1 --2- 172.21.15.57:0/1059820382 >> v2:172.21.15.80:3300/0 conn(0x7f7dbc074b50 0x7f7dbc0afff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.051+0000 7f7dbbfff640 1 -- 172.21.15.57:0/1059820382 >> v1:172.21.15.80:6789/0 conn(0x7f7dbc0b0530 legacy=0x7f7dbc06fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:02.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.051+0000 7f7dbbfff640 1 -- 172.21.15.57:0/1059820382 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dbc149da0 con 0x7f7dbc14a2c0 2024-04-25T23:49:02.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.051+0000 7f2654ff9640 1 -- 172.21.15.57:0/169899207 <== mon.2 v2:172.21.15.57:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2658154570 con 0x7f26580700d0 2024-04-25T23:49:02.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.055+0000 7f7dbbfff640 1 --2- 172.21.15.57:0/1059820382 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f7dbc14a2c0 0x7f7dbc14a710 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7db00097c0 tx=0x7f7db001bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6582aa1b34faf00 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:02.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.055+0000 7f7dba7fc640 1 -- 172.21.15.57:0/1059820382 <== mon.2 v2:172.21.15.57:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db00aa600 con 0x7f7dbc14a2c0 2024-04-25T23:49:02.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.055+0000 7f7dba7fc640 1 -- 172.21.15.57:0/1059820382 <== mon.2 v2:172.21.15.57:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7db00af070 con 0x7f7dbc14a2c0 2024-04-25T23:49:02.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.055+0000 7f7dc0989640 1 -- 172.21.15.57:0/1059820382 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f7dbc14a2c0 msgr2=0x7f7dbc14a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:02.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.055+0000 7f7dc0989640 1 --2- 172.21.15.57:0/1059820382 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f7dbc14a2c0 0x7f7dbc14a710 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7db00097c0 tx=0x7f7db001bf60 comp rx=0 tx=0).stop 2024-04-25T23:49:02.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.055+0000 7f7dc0989640 1 -- 172.21.15.57:0/1059820382 shutdown_connections 2024-04-25T23:49:02.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.055+0000 7f7dc0989640 1 --2- 172.21.15.57:0/1059820382 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f7dbc14a2c0 0x7f7dbc14a710 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.055+0000 7f7dc0989640 1 --2- 172.21.15.57:0/1059820382 >> v2:172.21.15.80:3300/0 conn(0x7f7dbc074b50 0x7f7dbc0afff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.055+0000 7f7dc0989640 1 -- 172.21.15.57:0/1059820382 >> 172.21.15.57:0/1059820382 conn(0x7f7dbc0b4f30 msgr2=0x7f7dbc0b5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:49:02.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.055+0000 7f7dc0989640 1 -- 172.21.15.57:0/1059820382 shutdown_connections 2024-04-25T23:49:02.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.055+0000 7f7dc0989640 1 -- 172.21.15.57:0/1059820382 wait complete. 2024-04-25T23:49:02.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.059+0000 7f7dc0989640 1 Processor -- start 2024-04-25T23:49:02.047 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.059+0000 7f7dc0989640 1 -- start start 2024-04-25T23:49:02.047 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.059+0000 7f7dc0989640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f7dbc074b50 0x7f7dbc0adf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:02.047 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.059+0000 7f7dc0989640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7dbc0b0530 0x7f7dbc0aacc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:02.047 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.059+0000 7f7dc0989640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7dbc0ae440 0x7f7dbc0ab200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:02.047 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.059+0000 7f7dc0989640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f7dbc11f530 con 0x7f7dbc0b0530 2024-04-25T23:49:02.047 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.059+0000 7f7dc0989640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f7dbc11f3b0 con 0x7f7dbc0ae440 2024-04-25T23:49:02.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.059+0000 7f7dc0989640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f7dbc11f6b0 con 0x7f7dbc074b50 2024-04-25T23:49:02.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.059+0000 7f7dbb7fe640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f7dbc074b50 0x7f7dbc0adf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:02.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.059+0000 7f7dbaffd640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7dbc0b0530 0x7f7dbc0aacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:02.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.059+0000 7f7dbb7fe640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f7dbc074b50 0x7f7dbc0adf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3301/0 says I am v2:172.21.15.57:47538/0 (socket says 172.21.15.57:47538) 2024-04-25T23:49:02.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.059+0000 7f7dbaffd640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7dbc0b0530 0x7f7dbc0aacc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33592/0 (socket says 172.21.15.57:33592) 2024-04-25T23:49:02.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.059+0000 7f7dbb7fe640 1 -- 172.21.15.57:0/3648064798 learned_addr learned my addr 172.21.15.57:0/3648064798 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:49:02.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.059+0000 7f7dbb7fe640 1 -- 172.21.15.57:0/3648064798 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7dbc0ae440 msgr2=0x7f7dbc0ab200 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T23:49:02.049 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.059+0000 7f7dbb7fe640 1 --2- 172.21.15.57:0/3648064798 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7dbc0ae440 0x7f7dbc0ab200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.049 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.059+0000 7f7dbb7fe640 1 -- 172.21.15.57:0/3648064798 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7dbc0b0530 msgr2=0x7f7dbc0aacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:02.049 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.059+0000 7f7dbb7fe640 1 --2- 172.21.15.57:0/3648064798 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7dbc0b0530 0x7f7dbc0aacc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.049 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.059+0000 7f7dbb7fe640 1 -- 172.21.15.57:0/3648064798 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dbc0ab740 con 0x7f7dbc074b50 2024-04-25T23:49:02.049 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.059+0000 7f7dbb7fe640 1 --2- 172.21.15.57:0/3648064798 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f7dbc074b50 0x7f7dbc0adf00 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7dac00d800 tx=0x7f7dac00dcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:02.049 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.063+0000 7f7da3fff640 1 -- 172.21.15.57:0/3648064798 <== mon.2 v2:172.21.15.57:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dac018020 con 0x7f7dbc074b50 2024-04-25T23:49:02.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.063+0000 7f7dc0989640 1 -- 172.21.15.57:0/3648064798 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dbc1319a0 con 0x7f7dbc074b50 2024-04-25T23:49:02.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.063+0000 7f7dc0989640 1 -- 172.21.15.57:0/3648064798 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dbc0ad6b0 con 0x7f7dbc074b50 2024-04-25T23:49:02.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.063+0000 7f7da3fff640 1 -- 172.21.15.57:0/3648064798 <== mon.2 v2:172.21.15.57:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dac0096a0 con 0x7f7dbc074b50 2024-04-25T23:49:02.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.063+0000 7f7da3fff640 1 -- 172.21.15.57:0/3648064798 <== mon.2 v2:172.21.15.57:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dac009a50 con 0x7f7dbc074b50 2024-04-25T23:49:02.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.063+0000 7f7dc0989640 1 -- 172.21.15.57:0/3648064798 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d9c005090 con 0x7f7dbc074b50 2024-04-25T23:49:02.054 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.067+0000 7f7da3fff640 1 -- 172.21.15.57:0/3648064798 <== mon.2 v2:172.21.15.57:3301/0 4 ==== mgrmap(e 5) ==== 87410+0+0 (secure 0 0 0) 0x7f7dac01b020 con 0x7f7dbc074b50 2024-04-25T23:49:02.054 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.067+0000 7f7da3fff640 1 --2- 172.21.15.57:0/3648064798 >> v2:172.21.15.80:6816/2493457677 conn(0x7f7da8067980 0x7f7da8069e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:02.054 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.067+0000 7f7da3fff640 1 -- 172.21.15.57:0/3648064798 <== mon.2 v2:172.21.15.57:3301/0 5 ==== osd_map(17..17 src has 1..17) ==== 4090+0+0 (secure 0 0 0) 0x7f7dac085970 con 0x7f7dbc074b50 2024-04-25T23:49:02.057 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.067+0000 7f7dbaffd640 1 --2- 172.21.15.57:0/3648064798 >> v2:172.21.15.80:6816/2493457677 conn(0x7f7da8067980 0x7f7da8069e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:02.057 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.067+0000 7f7dbaffd640 1 --2- 172.21.15.57:0/3648064798 >> v2:172.21.15.80:6816/2493457677 conn(0x7f7da8067980 0x7f7da8069e40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc144380 tx=0x7f7db4001820 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:02.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.075+0000 7f7da3fff640 1 -- 172.21.15.57:0/3648064798 <== mon.2 v2:172.21.15.57:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7dac022020 con 0x7f7dbc074b50 2024-04-25T23:49:02.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.091+0000 7f90b3952640 1 Processor -- start 2024-04-25T23:49:02.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.091+0000 7f90b3952640 1 -- start start 2024-04-25T23:49:02.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.091+0000 7f90b3952640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f90a4034c20 0x7f90a40315e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:02.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.091+0000 7f90b3952640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f90a40cfbf0 0x7f90a40c9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:02.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.091+0000 7f90b3952640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90a40d0080 con 0x7f90a40c9240 2024-04-25T23:49:02.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.091+0000 7f90b3952640 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 -- 0x7f90a4031b20 con 0x7f90a40cfbf0 2024-04-25T23:49:02.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.091+0000 7f90b3952640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f90a4035000 con 0x7f90a4034c20 2024-04-25T23:49:02.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.091+0000 7f90b1eef640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f90a40cfbf0 0x7f90a40c9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:02.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.091+0000 7f90b1eef640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f90a40cfbf0 0x7f90a40c9680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33606/0 (socket says 172.21.15.57:33606) 2024-04-25T23:49:02.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.091+0000 7f90b1eef640 1 -- 172.21.15.57:0/1092907365 learned_addr learned my addr 172.21.15.57:0/1092907365 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:49:02.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.091+0000 7f90b0eed640 1 --1- 172.21.15.57:0/1092907365 >> v1:172.21.15.57:6789/0 conn(0x7f90a40c9240 0x7f90a40cf4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:49694/0 (socket says 172.21.15.57:49694) 2024-04-25T23:49:02.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.091+0000 7f90b16ee640 1 --2- 172.21.15.57:0/1092907365 >> v2:172.21.15.80:3300/0 conn(0x7f90a4034c20 0x7f90a40315e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:02.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.091+0000 7f90a3fff640 1 -- 172.21.15.57:0/1092907365 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3228389860 0 0) 0x7f90ac04fa30 con 0x7f90a40c9240 2024-04-25T23:49:02.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.091+0000 7f90b1eef640 1 -- 172.21.15.57:0/1092907365 >> v2:172.21.15.80:3300/0 conn(0x7f90a4034c20 msgr2=0x7f90a40315e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:49:02.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.091+0000 7f90b1eef640 1 --2- 172.21.15.57:0/1092907365 >> v2:172.21.15.80:3300/0 conn(0x7f90a4034c20 0x7f90a40315e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.091+0000 7f90b1eef640 1 -- 172.21.15.57:0/1092907365 >> v1:172.21.15.57:6789/0 conn(0x7f90a40c9240 legacy=0x7f90a40cf4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:02.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.091+0000 7f90b1eef640 1 -- 172.21.15.57:0/1092907365 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90a4033200 con 0x7f90a40cfbf0 2024-04-25T23:49:02.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.091+0000 7f469525c640 1 Processor -- start 2024-04-25T23:49:02.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.091+0000 7f469525c640 1 -- start start 2024-04-25T23:49:02.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.095+0000 7f469525c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f46900b51f0 0x7f46900a9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:02.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.095+0000 7f469525c640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f46901429c0 0x7f4690142e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:02.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.095+0000 7f469525c640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4690143340 con 0x7f4690145d10 2024-04-25T23:49:02.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.095+0000 7f469525c640 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 -- 0x7f46901468b0 con 0x7f46901429c0 2024-04-25T23:49:02.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.095+0000 7f469525c640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f46900b55d0 con 0x7f46900b51f0 2024-04-25T23:49:02.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.095+0000 7f468ffff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f46900b51f0 0x7f46900a9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:02.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.095+0000 7f468ffff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f46900b51f0 0x7f46900a9650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.57:54416/0 (socket says 172.21.15.57:54416) 2024-04-25T23:49:02.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.095+0000 7f468ffff640 1 -- 172.21.15.57:0/2187509042 learned_addr learned my addr 172.21.15.57:0/2187509042 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:49:02.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.095+0000 7f468ffff640 1 -- 172.21.15.57:0/2187509042 >> v2:172.21.15.57:3300/0 conn(0x7f46901429c0 msgr2=0x7f4690142e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:49:02.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.095+0000 7f468ffff640 1 --2- 172.21.15.57:0/2187509042 >> v2:172.21.15.57:3300/0 conn(0x7f46901429c0 0x7f4690142e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.095+0000 7f468ffff640 1 -- 172.21.15.57:0/2187509042 >> v1:172.21.15.80:6789/0 conn(0x7f4690145d10 legacy=0x7f46901461a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:02.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.095+0000 7f468ffff640 1 -- 172.21.15.57:0/2187509042 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46900b48f0 con 0x7f46900b51f0 2024-04-25T23:49:02.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.095+0000 7f90b1eef640 1 --2- 172.21.15.57:0/1092907365 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90a40cfbf0 0x7f90a40c9680 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f90a4006a70 tx=0x7f90a801ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbe798a2d410560d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:02.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.095+0000 7f4db27b7640 1 Processor -- start 2024-04-25T23:49:02.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.095+0000 7f4db27b7640 1 -- start start 2024-04-25T23:49:02.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.095+0000 7f4db27b7640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f4dac06f4b0 0x7f4dac13bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:02.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.095+0000 7f4db27b7640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4dac070520 con 0x7f4dac073430 2024-04-25T23:49:02.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.095+0000 7f4db27b7640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4dac13ed50 con 0x7f4dac070140 2024-04-25T23:49:02.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.095+0000 7f4db27b7640 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 -- 0x7f4dac0708e0 con 0x7f4dac06f4b0 2024-04-25T23:49:02.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.095+0000 7f4dab7fe640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f4dac06f4b0 0x7f4dac13bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:02.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.095+0000 7f4dab7fe640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f4dac06f4b0 0x7f4dac13bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33620/0 (socket says 172.21.15.57:33620) 2024-04-25T23:49:02.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.095+0000 7f4dab7fe640 1 -- 172.21.15.57:0/2870904462 learned_addr learned my addr 172.21.15.57:0/2870904462 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:49:02.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.095+0000 7f90a3fff640 1 -- 172.21.15.57:0/1092907365 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a80b5070 con 0x7f90a40cfbf0 2024-04-25T23:49:02.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.095+0000 7f90a3fff640 1 -- 172.21.15.57:0/1092907365 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90a80a89f0 con 0x7f90a40cfbf0 2024-04-25T23:49:02.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.095+0000 7f90a3fff640 1 -- 172.21.15.57:0/1092907365 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a80a8dc0 con 0x7f90a40cfbf0 2024-04-25T23:49:02.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.095+0000 7f4dab7fe640 1 -- 172.21.15.57:0/2870904462 >> v1:172.21.15.80:6789/0 conn(0x7f4dac070140 legacy=0x7f4dac13b7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T23:49:02.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.095+0000 7f4dab7fe640 1 -- 172.21.15.57:0/2870904462 >> v1:172.21.15.57:6789/0 conn(0x7f4dac073430 legacy=0x7f4dac13e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:02.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.095+0000 7f4dab7fe640 1 -- 172.21.15.57:0/2870904462 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dac13ef20 con 0x7f4dac06f4b0 2024-04-25T23:49:02.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.099+0000 7f468ffff640 1 --2- 172.21.15.57:0/2187509042 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f46900b51f0 0x7f46900a9650 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f46900b2a30 tx=0x7f468401c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3df7ee89c8f1286 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:02.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.099+0000 7f90b3952640 1 -- 172.21.15.57:0/1092907365 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90a40cfbf0 msgr2=0x7f90a40c9680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:02.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.099+0000 7f90b3952640 1 --2- 172.21.15.57:0/1092907365 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90a40cfbf0 0x7f90a40c9680 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f90a4006a70 tx=0x7f90a801ecf0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.099+0000 7f90b3952640 1 -- 172.21.15.57:0/1092907365 shutdown_connections 2024-04-25T23:49:02.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.099+0000 7f90b3952640 1 --2- 172.21.15.57:0/1092907365 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90a40cfbf0 0x7f90a40c9680 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.099+0000 7f90b3952640 1 --2- 172.21.15.57:0/1092907365 >> v2:172.21.15.80:3300/0 conn(0x7f90a4034c20 0x7f90a40315e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.099+0000 7f90b3952640 1 -- 172.21.15.57:0/1092907365 >> 172.21.15.57:0/1092907365 conn(0x7f90a4005ef0 msgr2=0x7f90a40062d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:49:02.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.099+0000 7f468effd640 1 -- 172.21.15.57:0/2187509042 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46840b0070 con 0x7f46900b51f0 2024-04-25T23:49:02.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.099+0000 7f90b3952640 1 -- 172.21.15.57:0/1092907365 shutdown_connections 2024-04-25T23:49:02.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.099+0000 7f90b3952640 1 -- 172.21.15.57:0/1092907365 wait complete. 2024-04-25T23:49:02.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.099+0000 7f468effd640 1 -- 172.21.15.57:0/2187509042 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46840ab600 con 0x7f46900b51f0 2024-04-25T23:49:02.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.099+0000 7f468effd640 1 -- 172.21.15.57:0/2187509042 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46840ab9d0 con 0x7f46900b51f0 2024-04-25T23:49:02.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.099+0000 7f469525c640 1 -- 172.21.15.57:0/2187509042 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f46900b51f0 msgr2=0x7f46900a9650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:02.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.099+0000 7f469525c640 1 --2- 172.21.15.57:0/2187509042 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f46900b51f0 0x7f46900a9650 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f46900b2a30 tx=0x7f468401c3e0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.099+0000 7f469525c640 1 -- 172.21.15.57:0/2187509042 shutdown_connections 2024-04-25T23:49:02.089 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.099+0000 7f90b3952640 1 Processor -- start 2024-04-25T23:49:02.089 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.099+0000 7f90b3952640 1 -- start start 2024-04-25T23:49:02.089 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.099+0000 7f469525c640 1 --2- 172.21.15.57:0/2187509042 >> v2:172.21.15.57:3300/0 conn(0x7f46901429c0 0x7f4690142e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.099+0000 7f469525c640 1 --2- 172.21.15.57:0/2187509042 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f46900b51f0 0x7f46900a9650 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.099+0000 7f469525c640 1 -- 172.21.15.57:0/2187509042 >> 172.21.15.57:0/2187509042 conn(0x7f4690074ac0 msgr2=0x7f4690074ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:49:02.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.099+0000 7f469525c640 1 -- 172.21.15.57:0/2187509042 shutdown_connections 2024-04-25T23:49:02.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.099+0000 7f469525c640 1 -- 172.21.15.57:0/2187509042 wait complete. 2024-04-25T23:49:02.091 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.099+0000 7f90b3952640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90a4034c20 0x7f90a40c5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:02.091 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.099+0000 7f90b3952640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f90a40c9240 0x7f90a4164c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:02.091 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.099+0000 7f90b3952640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f90a40cfbf0 0x7f90a40ce8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:02.091 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.099+0000 7f90b3952640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f90a4031d40 con 0x7f90a4034c20 2024-04-25T23:49:02.091 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.099+0000 7f90b3952640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f90a40c9bc0 con 0x7f90a40cfbf0 2024-04-25T23:49:02.091 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.099+0000 7f90b3952640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f90a40d6870 con 0x7f90a40c9240 2024-04-25T23:49:02.091 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.099+0000 7f90b0eed640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f90a40c9240 0x7f90a4164c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:02.091 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.099+0000 7f90b16ee640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90a4034c20 0x7f90a40c5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:02.091 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.099+0000 7f90b0eed640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f90a40c9240 0x7f90a4164c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3301/0 says I am v2:172.21.15.57:47542/0 (socket says 172.21.15.57:47542) 2024-04-25T23:49:02.091 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.099+0000 7f90b0eed640 1 -- 172.21.15.57:0/2419178661 learned_addr learned my addr 172.21.15.57:0/2419178661 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:49:02.091 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.099+0000 7f4dab7fe640 1 --2- 172.21.15.57:0/2870904462 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4dac06f4b0 0x7f4dac13bf00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4da400d370 tx=0x7f4da401ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0708a3f174bc79 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:02.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.103+0000 7f90b1eef640 1 --2- 172.21.15.57:0/2419178661 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f90a40cfbf0 0x7f90a40ce8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:02.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.103+0000 7f90b0eed640 1 -- 172.21.15.57:0/2419178661 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f90a40cfbf0 msgr2=0x7f90a40ce8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:02.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.103+0000 7f90b0eed640 1 --2- 172.21.15.57:0/2419178661 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f90a40cfbf0 0x7f90a40ce8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.103+0000 7f90b0eed640 1 -- 172.21.15.57:0/2419178661 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90a4034c20 msgr2=0x7f90a40c5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:02.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.103+0000 7f90b0eed640 1 --2- 172.21.15.57:0/2419178661 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90a4034c20 0x7f90a40c5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.103+0000 7f90b0eed640 1 -- 172.21.15.57:0/2419178661 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90a40cee00 con 0x7f90a40c9240 2024-04-25T23:49:02.093 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.103+0000 7f4daaffd640 1 -- 172.21.15.57:0/2870904462 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da40ad070 con 0x7f4dac06f4b0 2024-04-25T23:49:02.093 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.103+0000 7f4daaffd640 1 -- 172.21.15.57:0/2870904462 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4da40a7d30 con 0x7f4dac06f4b0 2024-04-25T23:49:02.093 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.103+0000 7f4daaffd640 1 -- 172.21.15.57:0/2870904462 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da40b1620 con 0x7f4dac06f4b0 2024-04-25T23:49:02.093 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.103+0000 7f90b0eed640 1 --2- 172.21.15.57:0/2419178661 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f90a40c9240 0x7f90a4164c80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f90ac000b70 tx=0x7f90ac0686a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:02.093 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.103+0000 7f469525c640 1 Processor -- start 2024-04-25T23:49:02.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.103+0000 7f4db27b7640 1 -- 172.21.15.57:0/2870904462 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4dac06f4b0 msgr2=0x7f4dac13bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:02.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.103+0000 7f4db27b7640 1 --2- 172.21.15.57:0/2870904462 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4dac06f4b0 0x7f4dac13bf00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4da400d370 tx=0x7f4da401ed50 comp rx=0 tx=0).stop 2024-04-25T23:49:02.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.103+0000 7f4db27b7640 1 -- 172.21.15.57:0/2870904462 shutdown_connections 2024-04-25T23:49:02.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.103+0000 7f4db27b7640 1 --2- 172.21.15.57:0/2870904462 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4dac06f4b0 0x7f4dac13bf00 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.103+0000 7f4db27b7640 1 -- 172.21.15.57:0/2870904462 >> 172.21.15.57:0/2870904462 conn(0x7f4dac0b4480 msgr2=0x7f4dac0b4890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:49:02.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.103+0000 7f4db27b7640 1 -- 172.21.15.57:0/2870904462 shutdown_connections 2024-04-25T23:49:02.095 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.103+0000 7f469525c640 1 -- start start 2024-04-25T23:49:02.095 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.103+0000 7f469525c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f46900b51f0 0x7f469006d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:02.095 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.103+0000 7f469525c640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f46901429c0 0x7f46900b8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:02.095 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.103+0000 7f469525c640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4690145d10 0x7f4690137c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:02.095 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.103+0000 7f469525c640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f469014f710 con 0x7f46901429c0 2024-04-25T23:49:02.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.103+0000 7f469525c640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f46900a9da0 con 0x7f46900b51f0 2024-04-25T23:49:02.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.103+0000 7f469525c640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f469014f890 con 0x7f4690145d10 2024-04-25T23:49:02.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.103+0000 7f4694a5b640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4690145d10 0x7f4690137c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:02.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.103+0000 7f4694a5b640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4690145d10 0x7f4690137c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3301/0 says I am v2:172.21.15.57:47550/0 (socket says 172.21.15.57:47550) 2024-04-25T23:49:02.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.103+0000 7f4694a5b640 1 -- 172.21.15.57:0/420308113 learned_addr learned my addr 172.21.15.57:0/420308113 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:49:02.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.103+0000 7f468f7fe640 1 --2- 172.21.15.57:0/420308113 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f46901429c0 0x7f46900b8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:02.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.103+0000 7f468ffff640 1 --2- 172.21.15.57:0/420308113 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f46900b51f0 0x7f469006d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:02.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.103+0000 7f4694a5b640 1 -- 172.21.15.57:0/420308113 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f46900b51f0 msgr2=0x7f469006d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:02.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.103+0000 7f4694a5b640 1 --2- 172.21.15.57:0/420308113 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f46900b51f0 0x7f469006d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.103+0000 7f4694a5b640 1 -- 172.21.15.57:0/420308113 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f46901429c0 msgr2=0x7f46900b8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:02.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.103+0000 7f4694a5b640 1 --2- 172.21.15.57:0/420308113 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f46901429c0 0x7f46900b8700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.103+0000 7f4694a5b640 1 -- 172.21.15.57:0/420308113 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4690068520 con 0x7f4690145d10 2024-04-25T23:49:02.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.103+0000 7f4694a5b640 1 --2- 172.21.15.57:0/420308113 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4690145d10 0x7f4690137c30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f468800c730 tx=0x7f468800cb30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:02.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.103+0000 7f468cff9640 1 -- 172.21.15.57:0/420308113 <== mon.2 v2:172.21.15.57:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4688016020 con 0x7f4690145d10 2024-04-25T23:49:02.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.103+0000 7f4db27b7640 1 -- 172.21.15.57:0/2870904462 wait complete. 2024-04-25T23:49:02.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.103+0000 7f468cff9640 1 -- 172.21.15.57:0/420308113 <== mon.2 v2:172.21.15.57:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4688006d60 con 0x7f4690145d10 2024-04-25T23:49:02.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.103+0000 7f468cff9640 1 -- 172.21.15.57:0/420308113 <== mon.2 v2:172.21.15.57:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4688003bc0 con 0x7f4690145d10 2024-04-25T23:49:02.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.103+0000 7f469525c640 1 -- 172.21.15.57:0/420308113 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46900b8f60 con 0x7f4690145d10 2024-04-25T23:49:02.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.103+0000 7f4db27b7640 1 Processor -- start 2024-04-25T23:49:02.098 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.103+0000 7f469525c640 1 -- 172.21.15.57:0/420308113 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f46900b8c40 con 0x7f4690145d10 2024-04-25T23:49:02.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.103+0000 7f4db27b7640 1 -- start start 2024-04-25T23:49:02.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.107+0000 7f4db27b7640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4dac06f4b0 0x7f4dac131d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:02.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.107+0000 7f4db27b7640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4dac070140 0x7f4dac132a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:02.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.107+0000 7f4db27b7640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4dac073430 0x7f4dac1e2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:02.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.107+0000 7f4db27b7640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f4dac14cad0 con 0x7f4dac06f4b0 2024-04-25T23:49:02.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.107+0000 7f4db27b7640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f4dac14c950 con 0x7f4dac073430 2024-04-25T23:49:02.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.107+0000 7f4db27b7640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f4dac14cc50 con 0x7f4dac070140 2024-04-25T23:49:02.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.107+0000 7f4dabfff640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4dac06f4b0 0x7f4dac131d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:02.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.107+0000 7f4db0d54640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4dac073430 0x7f4dac1e2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:02.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.107+0000 7f4db0d54640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4dac073430 0x7f4dac1e2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.57:54448/0 (socket says 172.21.15.57:54448) 2024-04-25T23:49:02.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.107+0000 7f4db0d54640 1 -- 172.21.15.57:0/4248253600 learned_addr learned my addr 172.21.15.57:0/4248253600 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:49:02.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.107+0000 7f4db0d54640 1 -- 172.21.15.57:0/4248253600 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4dac070140 msgr2=0x7f4dac132a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:02.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.107+0000 7f4db0d54640 1 --2- 172.21.15.57:0/4248253600 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4dac070140 0x7f4dac132a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.099 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.107+0000 7f4db0d54640 1 -- 172.21.15.57:0/4248253600 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4dac06f4b0 msgr2=0x7f4dac131d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:02.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.107+0000 7f4db0d54640 1 --2- 172.21.15.57:0/4248253600 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4dac06f4b0 0x7f4dac131d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.107+0000 7f4db0d54640 1 -- 172.21.15.57:0/4248253600 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dac0ab290 con 0x7f4dac073430 2024-04-25T23:49:02.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.107+0000 7f90a1ffb640 1 -- 172.21.15.57:0/2419178661 <== mon.2 v2:172.21.15.57:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90ac08f070 con 0x7f90a40c9240 2024-04-25T23:49:02.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.107+0000 7f4db0d54640 1 --2- 172.21.15.57:0/4248253600 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4dac073430 0x7f4dac1e2580 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c00c390 tx=0x7f4d9c00c790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:02.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.107+0000 7f4da8ff9640 1 -- 172.21.15.57:0/4248253600 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d9c017070 con 0x7f4dac073430 2024-04-25T23:49:02.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.107+0000 7f4da8ff9640 1 -- 172.21.15.57:0/4248253600 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d9c0052d0 con 0x7f4dac073430 2024-04-25T23:49:02.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.107+0000 7f4db27b7640 1 -- 172.21.15.57:0/4248253600 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dac0a9950 con 0x7f4dac073430 2024-04-25T23:49:02.100 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.107+0000 7f4db27b7640 1 -- 172.21.15.57:0/4248253600 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dac1e2ac0 con 0x7f4dac073430 2024-04-25T23:49:02.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.107+0000 7f4da8ff9640 1 -- 172.21.15.57:0/4248253600 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d9c003840 con 0x7f4dac073430 2024-04-25T23:49:02.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.107+0000 7f4db27b7640 1 -- 172.21.15.57:0/4248253600 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dac068a10 con 0x7f4dac073430 2024-04-25T23:49:02.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.111+0000 7f4da8ff9640 1 -- 172.21.15.57:0/4248253600 <== mon.1 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 87410+0+0 (secure 0 0 0) 0x7f4d9c01a070 con 0x7f4dac073430 2024-04-25T23:49:02.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.107+0000 7f468cff9640 1 -- 172.21.15.57:0/420308113 <== mon.2 v2:172.21.15.57:3301/0 4 ==== mgrmap(e 5) ==== 87410+0+0 (secure 0 0 0) 0x7f46880050d0 con 0x7f4690145d10 2024-04-25T23:49:02.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.107+0000 7f468cff9640 1 --2- 172.21.15.57:0/420308113 >> v2:172.21.15.80:6816/2493457677 conn(0x7f46780679d0 0x7f4678069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:02.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.107+0000 7f468cff9640 1 -- 172.21.15.57:0/420308113 <== mon.2 v2:172.21.15.57:3301/0 5 ==== osd_map(17..17 src has 1..17) ==== 4090+0+0 (secure 0 0 0) 0x7f4688085150 con 0x7f4690145d10 2024-04-25T23:49:02.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.107+0000 7f468ffff640 1 --2- 172.21.15.57:0/420308113 >> v2:172.21.15.80:6816/2493457677 conn(0x7f46780679d0 0x7f4678069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:02.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.107+0000 7f468ffff640 1 --2- 172.21.15.57:0/420308113 >> v2:172.21.15.80:6816/2493457677 conn(0x7f46780679d0 0x7f4678069e90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f46840ad000 tx=0x7f4684008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:02.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.111+0000 7f469525c640 1 -- 172.21.15.57:0/420308113 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f469004f510 con 0x7f4690145d10 2024-04-25T23:49:02.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.107+0000 7f90b3952640 1 -- 172.21.15.57:0/2419178661 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90a40cc150 con 0x7f90a40c9240 2024-04-25T23:49:02.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.107+0000 7f90b3952640 1 -- 172.21.15.57:0/2419178661 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f90a40cc6b0 con 0x7f90a40c9240 2024-04-25T23:49:02.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.107+0000 7f90b3952640 1 -- 172.21.15.57:0/2419178661 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90a40052d0 con 0x7f90a40c9240 2024-04-25T23:49:02.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.111+0000 7f4da8ff9640 1 --2- 172.21.15.57:0/4248253600 >> v2:172.21.15.80:6816/2493457677 conn(0x7f4d98067880 0x7f4d98069d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:02.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.111+0000 7f4da8ff9640 1 -- 172.21.15.57:0/4248253600 <== mon.1 v2:172.21.15.80:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4090+0+0 (secure 0 0 0) 0x7f4d9c084a50 con 0x7f4dac073430 2024-04-25T23:49:02.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.111+0000 7f4dabfff640 1 --2- 172.21.15.57:0/4248253600 >> v2:172.21.15.80:6816/2493457677 conn(0x7f4d98067880 0x7f4d98069d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:02.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.115+0000 7f4dabfff640 1 --2- 172.21.15.57:0/4248253600 >> v2:172.21.15.80:6816/2493457677 conn(0x7f4d98067880 0x7f4d98069d40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4da0009980 tx=0x7f4da00090e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:02.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.111+0000 7f90a1ffb640 1 -- 172.21.15.57:0/2419178661 <== mon.2 v2:172.21.15.57:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90ac0933b0 con 0x7f90a40c9240 2024-04-25T23:49:02.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.111+0000 7f90a1ffb640 1 -- 172.21.15.57:0/2419178661 <== mon.2 v2:172.21.15.57:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90ac0937a0 con 0x7f90a40c9240 2024-04-25T23:49:02.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.111+0000 7f90a1ffb640 1 -- 172.21.15.57:0/2419178661 <== mon.2 v2:172.21.15.57:3301/0 4 ==== mgrmap(e 5) ==== 87410+0+0 (secure 0 0 0) 0x7f90ac092070 con 0x7f90a40c9240 2024-04-25T23:49:02.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.115+0000 7f90a1ffb640 1 --2- 172.21.15.57:0/2419178661 >> v2:172.21.15.80:6816/2493457677 conn(0x7f9098067a80 0x7f9098069f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:02.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.115+0000 7f90a1ffb640 1 -- 172.21.15.57:0/2419178661 <== mon.2 v2:172.21.15.57:3301/0 5 ==== osd_map(17..17 src has 1..17) ==== 4090+0+0 (secure 0 0 0) 0x7f90ac100e50 con 0x7f90a40c9240 2024-04-25T23:49:02.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.115+0000 7f90b16ee640 1 --2- 172.21.15.57:0/2419178661 >> v2:172.21.15.80:6816/2493457677 conn(0x7f9098067a80 0x7f9098069f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:02.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.115+0000 7f90b16ee640 1 --2- 172.21.15.57:0/2419178661 >> v2:172.21.15.80:6816/2493457677 conn(0x7f9098067a80 0x7f9098069f40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f909c006aa0 tx=0x7f909c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:02.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.115+0000 7f468cff9640 1 -- 172.21.15.57:0/420308113 <== mon.2 v2:172.21.15.57:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f468805c080 con 0x7f4690145d10 2024-04-25T23:49:02.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.119+0000 7f90a1ffb640 1 -- 172.21.15.57:0/2419178661 <== mon.2 v2:172.21.15.57:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f90a40d0080 con 0x7f90a40c9240 2024-04-25T23:49:02.111 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.123+0000 7f4da8ff9640 1 -- 172.21.15.57:0/4248253600 <== mon.1 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4d9c021080 con 0x7f4dac073430 2024-04-25T23:49:02.117 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.131+0000 7f8770567640 1 Processor -- start 2024-04-25T23:49:02.117 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.131+0000 7f8770567640 1 -- start start 2024-04-25T23:49:02.118 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.131+0000 7f8770567640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f8768071c80 0x7f8768070260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:02.118 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.131+0000 7f8770567640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f876806f050 0x7f8768152ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:02.118 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.131+0000 7f8770567640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f876806f470 0x7f87681533e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:02.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.131+0000 7f8770567640 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 -- 0x7f87680735b0 con 0x7f876806f050 2024-04-25T23:49:02.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.131+0000 7f8770567640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f8768156140 con 0x7f8768071c80 2024-04-25T23:49:02.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.131+0000 7f8770567640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f87680712a0 con 0x7f876806f470 2024-04-25T23:49:02.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.131+0000 7f3c5d665640 1 Processor -- start 2024-04-25T23:49:02.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.131+0000 7f3c5d665640 1 -- start start 2024-04-25T23:49:02.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.131+0000 7f3c5d665640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f3c58070290 0x7f3c58072e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:02.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.131+0000 7f3c5d665640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3c58147120 0x7f3c58143d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:02.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.131+0000 7f876db02640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f876806f050 0x7f8768152ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:02.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.131+0000 7f876db02640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f876806f050 0x7f8768152ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33648/0 (socket says 172.21.15.57:33648) 2024-04-25T23:49:02.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.131+0000 7f876db02640 1 -- 172.21.15.57:0/1936496474 learned_addr learned my addr 172.21.15.57:0/1936496474 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:49:02.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.131+0000 7f3c5d665640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c580717f0 con 0x7f3c5806efe0 2024-04-25T23:49:02.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.131+0000 7f3c5d665640 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 -- 0x7f3c58070670 con 0x7f3c58070290 2024-04-25T23:49:02.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.131+0000 7f3c5d665640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f3c5806fa20 con 0x7f3c58147120 2024-04-25T23:49:02.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.131+0000 7f3c577fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3c58147120 0x7f3c58143d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:02.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.131+0000 7f3c577fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3c58147120 0x7f3c58143d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.57:54460/0 (socket says 172.21.15.57:54460) 2024-04-25T23:49:02.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.131+0000 7f3c577fe640 1 -- 172.21.15.57:0/1605762559 learned_addr learned my addr 172.21.15.57:0/1605762559 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:49:02.121 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.131+0000 7f876eb04640 1 --2- 172.21.15.57:0/1936496474 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f876806f470 0x7f87681533e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:02.121 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.131+0000 7f876db02640 1 -- 172.21.15.57:0/1936496474 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f876806f470 msgr2=0x7f87681533e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:49:02.121 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.131+0000 7f876db02640 1 --2- 172.21.15.57:0/1936496474 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f876806f470 0x7f87681533e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.121 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.131+0000 7f876db02640 1 -- 172.21.15.57:0/1936496474 >> v2:172.21.15.80:3300/0 conn(0x7f8768071c80 msgr2=0x7f8768070260 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T23:49:02.123 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.131+0000 7f3c577fe640 1 -- 172.21.15.57:0/1605762559 >> v2:172.21.15.57:3300/0 conn(0x7f3c58070290 msgr2=0x7f3c58072e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:49:02.123 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.131+0000 7f3c577fe640 1 --2- 172.21.15.57:0/1605762559 >> v2:172.21.15.57:3300/0 conn(0x7f3c58070290 0x7f3c58072e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.123 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.131+0000 7f3c577fe640 1 -- 172.21.15.57:0/1605762559 >> v1:172.21.15.80:6789/0 conn(0x7f3c5806efe0 legacy=0x7f3c58146a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:02.123 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.131+0000 7f3c577fe640 1 -- 172.21.15.57:0/1605762559 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c5806fba0 con 0x7f3c58147120 2024-04-25T23:49:02.123 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.131+0000 7f876db02640 1 --2- 172.21.15.57:0/1936496474 >> v2:172.21.15.80:3300/0 conn(0x7f8768071c80 0x7f8768070260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.123 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.131+0000 7f876db02640 1 -- 172.21.15.57:0/1936496474 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8768071580 con 0x7f876806f050 2024-04-25T23:49:02.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.135+0000 7f876eb04640 1 --2- 172.21.15.57:0/1936496474 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f876806f470 0x7f87681533e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T23:49:02.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.135+0000 7f876db02640 1 --2- 172.21.15.57:0/1936496474 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f876806f050 0x7f8768152ea0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f87680ae950 tx=0x7f875801c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8996b86f65f4b768 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:02.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.135+0000 7f876d301640 1 -- 172.21.15.57:0/1936496474 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87580b0070 con 0x7f876806f050 2024-04-25T23:49:02.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.135+0000 7f876d301640 1 -- 172.21.15.57:0/1936496474 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87580ab600 con 0x7f876806f050 2024-04-25T23:49:02.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.135+0000 7f876d301640 1 -- 172.21.15.57:0/1936496474 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87580aba10 con 0x7f876806f050 2024-04-25T23:49:02.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.135+0000 7f3c577fe640 1 --2- 172.21.15.57:0/1605762559 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c58147120 0x7f3c58143d50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3c5806eec0 tx=0x7f3c5001c7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b5ee05a963ce00a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:02.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.135+0000 7f3c55ffb640 1 -- 172.21.15.57:0/1605762559 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c500a5530 con 0x7f3c58147120 2024-04-25T23:49:02.125 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.135+0000 7f3c55ffb640 1 -- 172.21.15.57:0/1605762559 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c500a56d0 con 0x7f3c58147120 2024-04-25T23:49:02.125 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.135+0000 7f3c55ffb640 1 -- 172.21.15.57:0/1605762559 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c500a5aa0 con 0x7f3c58147120 2024-04-25T23:49:02.125 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.139+0000 7f3c5d665640 1 -- 172.21.15.57:0/1605762559 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c58147120 msgr2=0x7f3c58143d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:02.125 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.139+0000 7f3c5d665640 1 --2- 172.21.15.57:0/1605762559 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c58147120 0x7f3c58143d50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3c5806eec0 tx=0x7f3c5001c7a0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.135+0000 7f8770567640 1 -- 172.21.15.57:0/1936496474 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f876806f050 msgr2=0x7f8768152ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:02.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.135+0000 7f8770567640 1 --2- 172.21.15.57:0/1936496474 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f876806f050 0x7f8768152ea0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f87680ae950 tx=0x7f875801c540 comp rx=0 tx=0).stop 2024-04-25T23:49:02.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.135+0000 7f8770567640 1 -- 172.21.15.57:0/1936496474 shutdown_connections 2024-04-25T23:49:02.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.135+0000 7f8770567640 1 --2- 172.21.15.57:0/1936496474 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f876806f470 0x7f87681533e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.135+0000 7f8770567640 1 --2- 172.21.15.57:0/1936496474 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f876806f050 0x7f8768152ea0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.135+0000 7f8770567640 1 --2- 172.21.15.57:0/1936496474 >> v2:172.21.15.80:3300/0 conn(0x7f8768071c80 0x7f8768070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.135+0000 7f8770567640 1 -- 172.21.15.57:0/1936496474 >> 172.21.15.57:0/1936496474 conn(0x7f87680b4480 msgr2=0x7f87680b4890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:49:02.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.139+0000 7f8770567640 1 -- 172.21.15.57:0/1936496474 shutdown_connections 2024-04-25T23:49:02.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.139+0000 7f3c5d665640 1 -- 172.21.15.57:0/1605762559 shutdown_connections 2024-04-25T23:49:02.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.139+0000 7f3c5d665640 1 --2- 172.21.15.57:0/1605762559 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c58147120 0x7f3c58143d50 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.139+0000 7f3c5d665640 1 --2- 172.21.15.57:0/1605762559 >> v2:172.21.15.57:3300/0 conn(0x7f3c58070290 0x7f3c58072e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.139+0000 7f3c5d665640 1 -- 172.21.15.57:0/1605762559 >> 172.21.15.57:0/1605762559 conn(0x7f3c580b4480 msgr2=0x7f3c580b4890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:49:02.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.139+0000 7f3c5d665640 1 -- 172.21.15.57:0/1605762559 shutdown_connections 2024-04-25T23:49:02.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.139+0000 7f3c5d665640 1 -- 172.21.15.57:0/1605762559 wait complete. 2024-04-25T23:49:02.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.139+0000 7f3c5d665640 1 Processor -- start 2024-04-25T23:49:02.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.139+0000 7f8770567640 1 -- 172.21.15.57:0/1936496474 wait complete. 2024-04-25T23:49:02.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.139+0000 7f8770567640 1 Processor -- start 2024-04-25T23:49:02.129 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.139+0000 7f3c5d665640 1 -- start start 2024-04-25T23:49:02.129 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.139+0000 7f3c5d665640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c5806efe0 0x7f3c581e7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:02.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.139+0000 7f3c5d665640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c58070290 0x7f3c580b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:02.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.139+0000 7f3c5d665640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f3c581496e0 0x7f3c580b2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:02.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.139+0000 7f3c5d665640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f3c58151f40 con 0x7f3c58070290 2024-04-25T23:49:02.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.139+0000 7f3c5d665640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f3c58151dc0 con 0x7f3c5806efe0 2024-04-25T23:49:02.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.139+0000 7f3c5d665640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f3c581520c0 con 0x7f3c581496e0 2024-04-25T23:49:02.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.139+0000 7f3c567fc640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c58070290 0x7f3c580b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:02.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.139+0000 7f3c567fc640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c58070290 0x7f3c580b3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33674/0 (socket says 172.21.15.57:33674) 2024-04-25T23:49:02.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.139+0000 7f3c567fc640 1 -- 172.21.15.57:0/393937715 learned_addr learned my addr 172.21.15.57:0/393937715 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:49:02.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.139+0000 7f3c56ffd640 1 --2- 172.21.15.57:0/393937715 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c5806efe0 0x7f3c581e7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:02.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.143+0000 7f3c56ffd640 1 -- 172.21.15.57:0/393937715 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f3c581496e0 msgr2=0x7f3c580b2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:02.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.143+0000 7f3c577fe640 1 --2- 172.21.15.57:0/393937715 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f3c581496e0 0x7f3c580b2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:02.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.143+0000 7f3c56ffd640 1 --2- 172.21.15.57:0/393937715 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f3c581496e0 0x7f3c580b2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.143+0000 7f3c56ffd640 1 -- 172.21.15.57:0/393937715 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c58070290 msgr2=0x7f3c580b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:02.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.143+0000 7f3c56ffd640 1 --2- 172.21.15.57:0/393937715 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c58070290 0x7f3c580b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.143+0000 7f3c56ffd640 1 -- 172.21.15.57:0/393937715 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c5814ab20 con 0x7f3c5806efe0 2024-04-25T23:49:02.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.143+0000 7f3c577fe640 1 --2- 172.21.15.57:0/393937715 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f3c581496e0 0x7f3c580b2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T23:49:02.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.143+0000 7f3c56ffd640 1 --2- 172.21.15.57:0/393937715 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c5806efe0 0x7f3c581e7ff0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c00b3f0 tx=0x7f3c4c00b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:02.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.139+0000 7f8770567640 1 -- start start 2024-04-25T23:49:02.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.139+0000 7f8770567640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f876806f050 0x7f8768158bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:02.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.139+0000 7f8770567640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f876806f470 0x7f87681efeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:02.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.139+0000 7f8770567640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8768071c80 0x7f87681f03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:02.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.139+0000 7f876db02640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f876806f470 0x7f87681efeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:02.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.139+0000 7f876db02640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f876806f470 0x7f87681efeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3301/0 says I am v2:172.21.15.57:47582/0 (socket says 172.21.15.57:47582) 2024-04-25T23:49:02.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.139+0000 7f876db02640 1 -- 172.21.15.57:0/1358678227 learned_addr learned my addr 172.21.15.57:0/1358678227 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:49:02.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.139+0000 7f8770567640 1 -- 172.21.15.57:0/1358678227 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f87680af710 con 0x7f876806f050 2024-04-25T23:49:02.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.139+0000 7f8770567640 1 -- 172.21.15.57:0/1358678227 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8768153d30 con 0x7f8768071c80 2024-04-25T23:49:02.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.139+0000 7f8770567640 1 -- 172.21.15.57:0/1358678227 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f87680af890 con 0x7f876806f470 2024-04-25T23:49:02.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.139+0000 7f876eb04640 1 --2- 172.21.15.57:0/1358678227 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8768071c80 0x7f87681f03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:02.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.139+0000 7f876eb04640 1 -- 172.21.15.57:0/1358678227 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8768071c80 msgr2=0x7f87681f03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-04-25T23:49:02.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.139+0000 7f876eb04640 1 -- 172.21.15.57:0/1358678227 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8768071c80 msgr2=0x7f87681f03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T23:49:02.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.139+0000 7f876eb04640 1 --2- 172.21.15.57:0/1358678227 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8768071c80 0x7f87681f03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T23:49:02.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.139+0000 7f876eb04640 1 --2- 172.21.15.57:0/1358678227 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8768071c80 0x7f87681f03f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T23:49:02.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.139+0000 7f876db02640 1 -- 172.21.15.57:0/1358678227 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8768071c80 msgr2=0x7f87681f03f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T23:49:02.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.139+0000 7f876db02640 1 --2- 172.21.15.57:0/1358678227 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8768071c80 0x7f87681f03f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.139+0000 7f876db02640 1 -- 172.21.15.57:0/1358678227 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f876806f050 msgr2=0x7f8768158bd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T23:49:02.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.139+0000 7f876db02640 1 --2- 172.21.15.57:0/1358678227 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f876806f050 0x7f8768158bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.139+0000 7f876db02640 1 -- 172.21.15.57:0/1358678227 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8768159110 con 0x7f876806f470 2024-04-25T23:49:02.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.143+0000 7f876db02640 1 --2- 172.21.15.57:0/1358678227 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f876806f470 0x7f87681efeb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f875801a170 tx=0x7f87580192e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:02.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.143+0000 7f875effd640 1 -- 172.21.15.57:0/1358678227 <== mon.2 v2:172.21.15.57:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87580b0050 con 0x7f876806f470 2024-04-25T23:49:02.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.143+0000 7f875effd640 1 -- 172.21.15.57:0/1358678227 <== mon.2 v2:172.21.15.57:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87580ba070 con 0x7f876806f470 2024-04-25T23:49:02.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.143+0000 7f875effd640 1 -- 172.21.15.57:0/1358678227 <== mon.2 v2:172.21.15.57:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87580b47b0 con 0x7f876806f470 2024-04-25T23:49:02.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.143+0000 7f8770567640 1 -- 172.21.15.57:0/1358678227 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f876808b1e0 con 0x7f876806f470 2024-04-25T23:49:02.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.143+0000 7f8770567640 1 -- 172.21.15.57:0/1358678227 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8768159f50 con 0x7f876806f470 2024-04-25T23:49:02.134 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.143+0000 7f875effd640 1 -- 172.21.15.57:0/1358678227 <== mon.2 v2:172.21.15.57:3301/0 4 ==== mgrmap(e 5) ==== 87410+0+0 (secure 0 0 0) 0x7f87580b4950 con 0x7f876806f470 2024-04-25T23:49:02.134 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.143+0000 7f875effd640 1 --2- 172.21.15.57:0/1358678227 >> v2:172.21.15.80:6816/2493457677 conn(0x7f874c0678d0 0x7f874c069d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:02.134 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.143+0000 7f875effd640 1 -- 172.21.15.57:0/1358678227 <== mon.2 v2:172.21.15.57:3301/0 5 ==== osd_map(17..17 src has 1..17) ==== 4090+0+0 (secure 0 0 0) 0x7f875811f400 con 0x7f876806f470 2024-04-25T23:49:02.134 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.143+0000 7f876e303640 1 --2- 172.21.15.57:0/1358678227 >> v2:172.21.15.80:6816/2493457677 conn(0x7f874c0678d0 0x7f874c069d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:02.134 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.147+0000 7f8770567640 1 -- 172.21.15.57:0/1358678227 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8750005090 con 0x7f876806f470 2024-04-25T23:49:02.134 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.147+0000 7f876e303640 1 --2- 172.21.15.57:0/1358678227 >> v2:172.21.15.80:6816/2493457677 conn(0x7f874c0678d0 0x7f874c069d90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8764009690 tx=0x7f8764006be0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:02.138 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.151+0000 7f875effd640 1 -- 172.21.15.57:0/1358678227 <== mon.2 v2:172.21.15.57:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8758016130 con 0x7f876806f470 2024-04-25T23:49:02.138 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.151+0000 7f3c3f7fe640 1 -- 172.21.15.57:0/393937715 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c015020 con 0x7f3c5806efe0 2024-04-25T23:49:02.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.151+0000 7f3c5d665640 1 -- 172.21.15.57:0/393937715 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c581e8530 con 0x7f3c5806efe0 2024-04-25T23:49:02.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.151+0000 7f3c5d665640 1 -- 172.21.15.57:0/393937715 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c5814ad90 con 0x7f3c5806efe0 2024-04-25T23:49:02.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.151+0000 7f3c3f7fe640 1 -- 172.21.15.57:0/393937715 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c4c011830 con 0x7f3c5806efe0 2024-04-25T23:49:02.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.151+0000 7f3c3f7fe640 1 -- 172.21.15.57:0/393937715 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c011c20 con 0x7f3c5806efe0 2024-04-25T23:49:02.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.151+0000 7f3c5d665640 1 -- 172.21.15.57:0/393937715 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c38005090 con 0x7f3c5806efe0 2024-04-25T23:49:02.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.155+0000 7f3c3f7fe640 1 -- 172.21.15.57:0/393937715 <== mon.1 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 87410+0+0 (secure 0 0 0) 0x7f3c4c018020 con 0x7f3c5806efe0 2024-04-25T23:49:02.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.155+0000 7f3c3f7fe640 1 --2- 172.21.15.57:0/393937715 >> v2:172.21.15.80:6816/2493457677 conn(0x7f3c44067980 0x7f3c44069e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:02.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.155+0000 7f3c3f7fe640 1 -- 172.21.15.57:0/393937715 <== mon.1 v2:172.21.15.80:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4090+0+0 (secure 0 0 0) 0x7f3c4c0859c0 con 0x7f3c5806efe0 2024-04-25T23:49:02.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.155+0000 7f3c567fc640 1 --2- 172.21.15.57:0/393937715 >> v2:172.21.15.80:6816/2493457677 conn(0x7f3c44067980 0x7f3c44069e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:02.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.155+0000 7f3c567fc640 1 --2- 172.21.15.57:0/393937715 >> v2:172.21.15.80:6816/2493457677 conn(0x7f3c44067980 0x7f3c44069e40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3c5814a9e0 tx=0x7f3c48009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:02.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.167+0000 7f3c3f7fe640 1 -- 172.21.15.57:0/393937715 <== mon.1 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3c4c01f020 con 0x7f3c5806efe0 2024-04-25T23:49:02.346 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.359+0000 7fcb4b5ff640 1 -- 172.21.15.57:0/128034274 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fcb20005380 con 0x7fcb4406f1a0 2024-04-25T23:49:02.346 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.359+0000 7fcb39ffb640 1 -- 172.21.15.57:0/128034274 <== mon.2 v2:172.21.15.57:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fcb40016070 con 0x7fcb4406f1a0 2024-04-25T23:49:02.349 INFO:teuthology.orchestra.run.smithi057.stdout:55834574850 2024-04-25T23:49:02.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.363+0000 7fcb2f7fe640 1 -- 172.21.15.57:0/128034274 >> v2:172.21.15.80:6816/2493457677 conn(0x7fcb30067af0 msgr2=0x7fcb30069fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:02.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.363+0000 7fcb2f7fe640 1 --2- 172.21.15.57:0/128034274 >> v2:172.21.15.80:6816/2493457677 conn(0x7fcb30067af0 0x7fcb30069fb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c000f90 tx=0x7fcb3c009040 comp rx=0 tx=0).stop 2024-04-25T23:49:02.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.363+0000 7fcb2f7fe640 1 -- 172.21.15.57:0/128034274 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fcb4406f1a0 msgr2=0x7fcb441349f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:02.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.363+0000 7fcb2f7fe640 1 --2- 172.21.15.57:0/128034274 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fcb4406f1a0 0x7fcb441349f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fcb440706b0 tx=0x7fcb40019100 comp rx=0 tx=0).stop 2024-04-25T23:49:02.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.363+0000 7fcb2f7fe640 1 -- 172.21.15.57:0/128034274 shutdown_connections 2024-04-25T23:49:02.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.363+0000 7fcb2f7fe640 1 --2- 172.21.15.57:0/128034274 >> v2:172.21.15.80:6816/2493457677 conn(0x7fcb30067af0 0x7fcb30069fb0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.363+0000 7fcb2f7fe640 1 --2- 172.21.15.57:0/128034274 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcb4412d780 0x7fcb4413ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.363+0000 7fcb2f7fe640 1 --2- 172.21.15.57:0/128034274 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb4412d320 0x7fcb4413e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.363+0000 7fcb2f7fe640 1 --2- 172.21.15.57:0/128034274 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fcb4406f1a0 0x7fcb441349f0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.363+0000 7fcb2f7fe640 1 -- 172.21.15.57:0/128034274 >> 172.21.15.57:0/128034274 conn(0x7fcb440b4070 msgr2=0x7fcb440aedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:49:02.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.363+0000 7fcb2f7fe640 1 -- 172.21.15.57:0/128034274 shutdown_connections 2024-04-25T23:49:02.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.363+0000 7fcb2f7fe640 1 -- 172.21.15.57:0/128034274 wait complete. 2024-04-25T23:49:02.365 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.379+0000 7f265ea43640 1 -- 172.21.15.57:0/169899207 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f2634005650 con 0x7f26580700d0 2024-04-25T23:49:02.365 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.379+0000 7f2654ff9640 1 -- 172.21.15.57:0/169899207 <== mon.2 v2:172.21.15.57:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f264801a320 con 0x7f26580700d0 2024-04-25T23:49:02.368 INFO:teuthology.orchestra.run.smithi057.stdout:55834574850 2024-04-25T23:49:02.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.379+0000 7f26467fc640 1 -- 172.21.15.57:0/169899207 >> v2:172.21.15.80:6816/2493457677 conn(0x7f263c0679b0 msgr2=0x7f263c069e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:02.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.379+0000 7f26467fc640 1 --2- 172.21.15.57:0/169899207 >> v2:172.21.15.80:6816/2493457677 conn(0x7f263c0679b0 0x7f263c069e70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f26500012f0 tx=0x7f2650001770 comp rx=0 tx=0).stop 2024-04-25T23:49:02.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.379+0000 7f26467fc640 1 -- 172.21.15.57:0/169899207 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f26580700d0 msgr2=0x7f2658154d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:02.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.379+0000 7f26467fc640 1 --2- 172.21.15.57:0/169899207 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f26580700d0 0x7f2658154d10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f26480ae040 tx=0x7f2648009a20 comp rx=0 tx=0).stop 2024-04-25T23:49:02.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.383+0000 7f26467fc640 1 -- 172.21.15.57:0/169899207 shutdown_connections 2024-04-25T23:49:02.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.383+0000 7f26467fc640 1 --2- 172.21.15.57:0/169899207 >> v2:172.21.15.80:6816/2493457677 conn(0x7f263c0679b0 0x7f263c069e70 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.383+0000 7f26467fc640 1 --2- 172.21.15.57:0/169899207 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2658153500 0x7f26581568c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.383+0000 7f26467fc640 1 --2- 172.21.15.57:0/169899207 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f26580700d0 0x7f2658154d10 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.383+0000 7f26467fc640 1 --2- 172.21.15.57:0/169899207 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f265806efb0 0x7f2658154030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.383+0000 7f26467fc640 1 -- 172.21.15.57:0/169899207 >> 172.21.15.57:0/169899207 conn(0x7f26580b4070 msgr2=0x7f26580b1a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:49:02.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.383+0000 7f26467fc640 1 -- 172.21.15.57:0/169899207 shutdown_connections 2024-04-25T23:49:02.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.383+0000 7f26467fc640 1 -- 172.21.15.57:0/169899207 wait complete. 2024-04-25T23:49:02.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.391+0000 7f7dc0989640 1 -- 172.21.15.57:0/3648064798 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f7d9c005380 con 0x7f7dbc074b50 2024-04-25T23:49:02.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.391+0000 7f7da3fff640 1 -- 172.21.15.57:0/3648064798 <== mon.2 v2:172.21.15.57:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7dac056c30 con 0x7f7dbc074b50 2024-04-25T23:49:02.378 INFO:teuthology.orchestra.run.smithi057.stdout:55834574850 2024-04-25T23:49:02.379 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574850 for osd.0 2024-04-25T23:49:02.379 DEBUG:teuthology.parallel:result is None 2024-04-25T23:49:02.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.391+0000 7f7da1ffb640 1 -- 172.21.15.57:0/3648064798 >> v2:172.21.15.80:6816/2493457677 conn(0x7f7da8067980 msgr2=0x7f7da8069e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:02.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.391+0000 7f7da1ffb640 1 --2- 172.21.15.57:0/3648064798 >> v2:172.21.15.80:6816/2493457677 conn(0x7f7da8067980 0x7f7da8069e40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc144380 tx=0x7f7db4001820 comp rx=0 tx=0).stop 2024-04-25T23:49:02.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.391+0000 7f7da1ffb640 1 -- 172.21.15.57:0/3648064798 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f7dbc074b50 msgr2=0x7f7dbc0adf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:02.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.391+0000 7f7da1ffb640 1 --2- 172.21.15.57:0/3648064798 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f7dbc074b50 0x7f7dbc0adf00 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7dac00d800 tx=0x7f7dac00dcc0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.391+0000 7f7da1ffb640 1 -- 172.21.15.57:0/3648064798 shutdown_connections 2024-04-25T23:49:02.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.391+0000 7f7da1ffb640 1 --2- 172.21.15.57:0/3648064798 >> v2:172.21.15.80:6816/2493457677 conn(0x7f7da8067980 0x7f7da8069e40 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.391+0000 7f7da1ffb640 1 --2- 172.21.15.57:0/3648064798 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7dbc0ae440 0x7f7dbc0ab200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.391+0000 7f7da1ffb640 1 --2- 172.21.15.57:0/3648064798 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7dbc0b0530 0x7f7dbc0aacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.391+0000 7f7da1ffb640 1 --2- 172.21.15.57:0/3648064798 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f7dbc074b50 0x7f7dbc0adf00 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.391+0000 7f7da1ffb640 1 -- 172.21.15.57:0/3648064798 >> 172.21.15.57:0/3648064798 conn(0x7f7dbc0b4f30 msgr2=0x7f7dbc06f880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:49:02.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.391+0000 7f7da1ffb640 1 -- 172.21.15.57:0/3648064798 shutdown_connections 2024-04-25T23:49:02.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.391+0000 7f7da1ffb640 1 -- 172.21.15.57:0/3648064798 wait complete. 2024-04-25T23:49:02.395 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574850 for osd.2 2024-04-25T23:49:02.395 DEBUG:teuthology.parallel:result is None 2024-04-25T23:49:02.398 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574850 for osd.1 2024-04-25T23:49:02.398 DEBUG:teuthology.parallel:result is None 2024-04-25T23:49:02.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.447+0000 7f3c5d665640 1 -- 172.21.15.57:0/393937715 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f3c38005650 con 0x7f3c5806efe0 2024-04-25T23:49:02.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.447+0000 7f3c3f7fe640 1 -- 172.21.15.57:0/393937715 <== mon.1 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3c4c056b30 con 0x7f3c5806efe0 2024-04-25T23:49:02.436 INFO:teuthology.orchestra.run.smithi057.stdout:60129542146 2024-04-25T23:49:02.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.451+0000 7f3c5d665640 1 -- 172.21.15.57:0/393937715 >> v2:172.21.15.80:6816/2493457677 conn(0x7f3c44067980 msgr2=0x7f3c44069e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:02.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.451+0000 7f3c5d665640 1 --2- 172.21.15.57:0/393937715 >> v2:172.21.15.80:6816/2493457677 conn(0x7f3c44067980 0x7f3c44069e40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3c5814a9e0 tx=0x7f3c48009040 comp rx=0 tx=0).stop 2024-04-25T23:49:02.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.451+0000 7f3c5d665640 1 -- 172.21.15.57:0/393937715 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c5806efe0 msgr2=0x7f3c581e7ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:02.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.451+0000 7f3c5d665640 1 --2- 172.21.15.57:0/393937715 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c5806efe0 0x7f3c581e7ff0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c00b3f0 tx=0x7f3c4c00b8b0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.451+0000 7f3c5d665640 1 -- 172.21.15.57:0/393937715 shutdown_connections 2024-04-25T23:49:02.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.451+0000 7f3c5d665640 1 --2- 172.21.15.57:0/393937715 >> v2:172.21.15.80:6816/2493457677 conn(0x7f3c44067980 0x7f3c44069e40 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.451+0000 7f3c5d665640 1 --2- 172.21.15.57:0/393937715 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f3c581496e0 0x7f3c580b2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.451+0000 7f3c5d665640 1 --2- 172.21.15.57:0/393937715 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c58070290 0x7f3c580b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.451+0000 7f3c5d665640 1 --2- 172.21.15.57:0/393937715 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c5806efe0 0x7f3c581e7ff0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.451+0000 7f3c5d665640 1 -- 172.21.15.57:0/393937715 >> 172.21.15.57:0/393937715 conn(0x7f3c580b4480 msgr2=0x7f3c580af0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:49:02.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.451+0000 7f3c5d665640 1 -- 172.21.15.57:0/393937715 shutdown_connections 2024-04-25T23:49:02.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.451+0000 7f3c5d665640 1 -- 172.21.15.57:0/393937715 wait complete. 2024-04-25T23:49:02.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.455+0000 7f469525c640 1 -- 172.21.15.57:0/420308113 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f46900b5d50 con 0x7f4690145d10 2024-04-25T23:49:02.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.455+0000 7f90b3952640 1 -- 172.21.15.57:0/2419178661 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f90a40d6650 con 0x7f90a40c9240 2024-04-25T23:49:02.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.455+0000 7f468cff9640 1 -- 172.21.15.57:0/420308113 <== mon.2 v2:172.21.15.57:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4688020020 con 0x7f4690145d10 2024-04-25T23:49:02.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.455+0000 7f90a1ffb640 1 -- 172.21.15.57:0/2419178661 <== mon.2 v2:172.21.15.57:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f90ac0d1970 con 0x7f90a40c9240 2024-04-25T23:49:02.445 INFO:teuthology.orchestra.run.smithi057.stdout:60129542147 2024-04-25T23:49:02.445 INFO:teuthology.orchestra.run.smithi057.stdout:60129542146 2024-04-25T23:49:02.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.459+0000 7f469525c640 1 -- 172.21.15.57:0/420308113 >> v2:172.21.15.80:6816/2493457677 conn(0x7f46780679d0 msgr2=0x7f4678069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:02.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.459+0000 7f469525c640 1 --2- 172.21.15.57:0/420308113 >> v2:172.21.15.80:6816/2493457677 conn(0x7f46780679d0 0x7f4678069e90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f46840ad000 tx=0x7f4684008040 comp rx=0 tx=0).stop 2024-04-25T23:49:02.448 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.459+0000 7f469525c640 1 -- 172.21.15.57:0/420308113 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4690145d10 msgr2=0x7f4690137c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:02.448 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.459+0000 7f469525c640 1 --2- 172.21.15.57:0/420308113 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4690145d10 0x7f4690137c30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f468800c730 tx=0x7f468800cb30 comp rx=0 tx=0).stop 2024-04-25T23:49:02.448 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.459+0000 7f90b3952640 1 -- 172.21.15.57:0/2419178661 >> v2:172.21.15.80:6816/2493457677 conn(0x7f9098067a80 msgr2=0x7f9098069f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:02.448 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.459+0000 7f90b3952640 1 --2- 172.21.15.57:0/2419178661 >> v2:172.21.15.80:6816/2493457677 conn(0x7f9098067a80 0x7f9098069f40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f909c006aa0 tx=0x7f909c008040 comp rx=0 tx=0).stop 2024-04-25T23:49:02.448 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.459+0000 7f90b3952640 1 -- 172.21.15.57:0/2419178661 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f90a40c9240 msgr2=0x7f90a4164c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:02.448 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.459+0000 7f90b3952640 1 --2- 172.21.15.57:0/2419178661 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f90a40c9240 0x7f90a4164c80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f90ac000b70 tx=0x7f90ac0686a0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.459+0000 7f469525c640 1 -- 172.21.15.57:0/420308113 shutdown_connections 2024-04-25T23:49:02.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.459+0000 7f90b3952640 1 -- 172.21.15.57:0/2419178661 shutdown_connections 2024-04-25T23:49:02.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.459+0000 7f90b3952640 1 --2- 172.21.15.57:0/2419178661 >> v2:172.21.15.80:6816/2493457677 conn(0x7f9098067a80 0x7f9098069f40 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.459+0000 7f90b3952640 1 --2- 172.21.15.57:0/2419178661 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f90a40cfbf0 0x7f90a40ce8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.459+0000 7f90b3952640 1 --2- 172.21.15.57:0/2419178661 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f90a40c9240 0x7f90a4164c80 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.459+0000 7f90b3952640 1 --2- 172.21.15.57:0/2419178661 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90a4034c20 0x7f90a40c5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.459+0000 7f90b3952640 1 -- 172.21.15.57:0/2419178661 >> 172.21.15.57:0/2419178661 conn(0x7f90a4005ef0 msgr2=0x7f90a40062d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:49:02.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.459+0000 7f90b3952640 1 -- 172.21.15.57:0/2419178661 shutdown_connections 2024-04-25T23:49:02.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.459+0000 7f469525c640 1 --2- 172.21.15.57:0/420308113 >> v2:172.21.15.80:6816/2493457677 conn(0x7f46780679d0 0x7f4678069e90 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.459+0000 7f90b3952640 1 -- 172.21.15.57:0/2419178661 wait complete. 2024-04-25T23:49:02.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.459+0000 7f469525c640 1 --2- 172.21.15.57:0/420308113 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4690145d10 0x7f4690137c30 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.459+0000 7f469525c640 1 --2- 172.21.15.57:0/420308113 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f46901429c0 0x7f46900b8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.459+0000 7f469525c640 1 --2- 172.21.15.57:0/420308113 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f46900b51f0 0x7f469006d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.459+0000 7f469525c640 1 -- 172.21.15.57:0/420308113 >> 172.21.15.57:0/420308113 conn(0x7f4690074ac0 msgr2=0x7f4690061c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:49:02.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.459+0000 7f469525c640 1 -- 172.21.15.57:0/420308113 shutdown_connections 2024-04-25T23:49:02.452 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.459+0000 7f469525c640 1 -- 172.21.15.57:0/420308113 wait complete. 2024-04-25T23:49:02.452 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.463+0000 7f4db27b7640 1 -- 172.21.15.57:0/4248253600 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f4dac0a9b20 con 0x7f4dac073430 2024-04-25T23:49:02.452 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.463+0000 7f4da8ff9640 1 -- 172.21.15.57:0/4248253600 <== mon.1 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4dac0a9b20 con 0x7f4dac073430 2024-04-25T23:49:02.452 INFO:teuthology.orchestra.run.smithi057.stdout:55834574850 2024-04-25T23:49:02.454 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.467+0000 7f4db27b7640 1 -- 172.21.15.57:0/4248253600 >> v2:172.21.15.80:6816/2493457677 conn(0x7f4d98067880 msgr2=0x7f4d98069d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:02.454 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.467+0000 7f4db27b7640 1 --2- 172.21.15.57:0/4248253600 >> v2:172.21.15.80:6816/2493457677 conn(0x7f4d98067880 0x7f4d98069d40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4da0009980 tx=0x7f4da00090e0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.454 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.467+0000 7f4db27b7640 1 -- 172.21.15.57:0/4248253600 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4dac073430 msgr2=0x7f4dac1e2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:02.454 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.467+0000 7f4db27b7640 1 --2- 172.21.15.57:0/4248253600 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4dac073430 0x7f4dac1e2580 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c00c390 tx=0x7f4d9c00c790 comp rx=0 tx=0).stop 2024-04-25T23:49:02.454 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.467+0000 7f4db27b7640 1 -- 172.21.15.57:0/4248253600 shutdown_connections 2024-04-25T23:49:02.454 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.467+0000 7f4db27b7640 1 --2- 172.21.15.57:0/4248253600 >> v2:172.21.15.80:6816/2493457677 conn(0x7f4d98067880 0x7f4d98069d40 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.455 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.467+0000 7f4db27b7640 1 --2- 172.21.15.57:0/4248253600 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4dac073430 0x7f4dac1e2580 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.455 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.467+0000 7f4db27b7640 1 --2- 172.21.15.57:0/4248253600 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f4dac070140 0x7f4dac132a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.455 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.467+0000 7f4db27b7640 1 --2- 172.21.15.57:0/4248253600 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4dac06f4b0 0x7f4dac131d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.455 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.467+0000 7f4db27b7640 1 -- 172.21.15.57:0/4248253600 >> 172.21.15.57:0/4248253600 conn(0x7f4dac0b4480 msgr2=0x7f4dac0b14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:49:02.455 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.467+0000 7f4db27b7640 1 -- 172.21.15.57:0/4248253600 shutdown_connections 2024-04-25T23:49:02.455 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.467+0000 7f4db27b7640 1 -- 172.21.15.57:0/4248253600 wait complete. 2024-04-25T23:49:02.456 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542146 got 60129542146 for osd.6 2024-04-25T23:49:02.456 DEBUG:teuthology.parallel:result is None 2024-04-25T23:49:02.458 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.471+0000 7f8770567640 1 -- 172.21.15.57:0/1358678227 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f8750005380 con 0x7f876806f470 2024-04-25T23:49:02.458 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.471+0000 7f875effd640 1 -- 172.21.15.57:0/1358678227 <== mon.2 v2:172.21.15.57:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f87580f0600 con 0x7f876806f470 2024-04-25T23:49:02.459 INFO:teuthology.orchestra.run.smithi057.stdout:55834574850 2024-04-25T23:49:02.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.471+0000 7f8770567640 1 -- 172.21.15.57:0/1358678227 >> v2:172.21.15.80:6816/2493457677 conn(0x7f874c0678d0 msgr2=0x7f874c069d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:02.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.471+0000 7f8770567640 1 --2- 172.21.15.57:0/1358678227 >> v2:172.21.15.80:6816/2493457677 conn(0x7f874c0678d0 0x7f874c069d90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8764009690 tx=0x7f8764006be0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.471+0000 7f8770567640 1 -- 172.21.15.57:0/1358678227 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f876806f470 msgr2=0x7f87681efeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:02.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.471+0000 7f8770567640 1 --2- 172.21.15.57:0/1358678227 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f876806f470 0x7f87681efeb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f875801a170 tx=0x7f87580192e0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.471+0000 7f8770567640 1 -- 172.21.15.57:0/1358678227 shutdown_connections 2024-04-25T23:49:02.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.471+0000 7f8770567640 1 --2- 172.21.15.57:0/1358678227 >> v2:172.21.15.80:6816/2493457677 conn(0x7f874c0678d0 0x7f874c069d90 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.471+0000 7f8770567640 1 --2- 172.21.15.57:0/1358678227 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8768071c80 0x7f87681f03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.471+0000 7f8770567640 1 --2- 172.21.15.57:0/1358678227 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f876806f470 0x7f87681efeb0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.471+0000 7f8770567640 1 --2- 172.21.15.57:0/1358678227 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f876806f050 0x7f8768158bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.471+0000 7f8770567640 1 -- 172.21.15.57:0/1358678227 >> 172.21.15.57:0/1358678227 conn(0x7f87680b4480 msgr2=0x7f8768061c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:49:02.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.471+0000 7f8770567640 1 -- 172.21.15.57:0/1358678227 shutdown_connections 2024-04-25T23:49:02.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.475+0000 7f8770567640 1 -- 172.21.15.57:0/1358678227 wait complete. 2024-04-25T23:49:02.470 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542146 got 60129542146 for osd.4 2024-04-25T23:49:02.470 DEBUG:teuthology.parallel:result is None 2024-04-25T23:49:02.478 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542147 got 60129542147 for osd.5 2024-04-25T23:49:02.478 DEBUG:teuthology.parallel:result is None 2024-04-25T23:49:02.480 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574850 for osd.7 2024-04-25T23:49:02.480 DEBUG:teuthology.parallel:result is None 2024-04-25T23:49:02.481 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574850 for osd.3 2024-04-25T23:49:02.482 DEBUG:teuthology.parallel:result is None 2024-04-25T23:49:02.482 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T23:49:02.482 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T23:49:02.630 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.643+0000 7f60e0945640 1 Processor -- start 2024-04-25T23:49:02.630 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.643+0000 7f60e0945640 1 -- start start 2024-04-25T23:49:02.631 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.643+0000 7f60e0945640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f60dc071c40 0x7f60dc1438f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:02.631 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.643+0000 7f60e0945640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60dc0725f0 con 0x7f60dc074b50 2024-04-25T23:49:02.631 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.643+0000 7f60e0945640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60dc0a94e0 con 0x7f60dc13dd40 2024-04-25T23:49:02.631 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.643+0000 7f60e0945640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f60dc074f30 con 0x7f60dc071c40 2024-04-25T23:49:02.631 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.643+0000 7f60d9d9b640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f60dc071c40 0x7f60dc1438f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:02.631 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.643+0000 7f60da59c640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f60dc074b50 0x7f60dc13d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:49710/0 (socket says 172.21.15.57:49710) 2024-04-25T23:49:02.631 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.643+0000 7f60da59c640 1 -- 172.21.15.57:0/669023308 learned_addr learned my addr 172.21.15.57:0/669023308 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-04-25T23:49:02.632 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.643+0000 7f60d959a640 1 -- 172.21.15.57:0/669023308 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2864549163 0 0) 0x7f60dc0725f0 con 0x7f60dc074b50 2024-04-25T23:49:02.632 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.643+0000 7f60d959a640 1 -- 172.21.15.57:0/669023308 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f60c8003530 con 0x7f60dc074b50 2024-04-25T23:49:02.632 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.643+0000 7f60d9d9b640 1 -- 172.21.15.57:0/669023308 >> v1:172.21.15.80:6789/0 conn(0x7f60dc13dd40 legacy=0x7f60dc14ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:02.632 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.643+0000 7f60d9d9b640 1 -- 172.21.15.57:0/669023308 >> v1:172.21.15.57:6789/0 conn(0x7f60dc074b50 legacy=0x7f60dc13d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:02.632 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.643+0000 7f60d9d9b640 1 -- 172.21.15.57:0/669023308 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60dc073750 con 0x7f60dc071c40 2024-04-25T23:49:02.634 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.647+0000 7f60d9d9b640 1 --2- 172.21.15.57:0/669023308 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f60dc071c40 0x7f60dc1438f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f60c4009dc0 tx=0x7f60c401c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d8d72d96ca9e5ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:02.634 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.647+0000 7f60d959a640 1 -- 172.21.15.57:0/669023308 <== mon.2 v2:172.21.15.57:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c40b2070 con 0x7f60dc071c40 2024-04-25T23:49:02.634 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.647+0000 7f60d959a640 1 -- 172.21.15.57:0/669023308 <== mon.2 v2:172.21.15.57:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60c40a58b0 con 0x7f60dc071c40 2024-04-25T23:49:02.634 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.647+0000 7f60d959a640 1 -- 172.21.15.57:0/669023308 <== mon.2 v2:172.21.15.57:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c40a5c80 con 0x7f60dc071c40 2024-04-25T23:49:02.635 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.647+0000 7f60e0945640 1 -- 172.21.15.57:0/669023308 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f60dc071c40 msgr2=0x7f60dc1438f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:02.635 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.647+0000 7f60e0945640 1 --2- 172.21.15.57:0/669023308 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f60dc071c40 0x7f60dc1438f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f60c4009dc0 tx=0x7f60c401c8e0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.635 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.647+0000 7f60e0945640 1 -- 172.21.15.57:0/669023308 shutdown_connections 2024-04-25T23:49:02.635 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.647+0000 7f60e0945640 1 --2- 172.21.15.57:0/669023308 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f60dc071c40 0x7f60dc1438f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.635 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.647+0000 7f60e0945640 1 -- 172.21.15.57:0/669023308 >> 172.21.15.57:0/669023308 conn(0x7f60dc0b5320 msgr2=0x7f60dc0b5730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:49:02.635 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.647+0000 7f60e0945640 1 -- 172.21.15.57:0/669023308 shutdown_connections 2024-04-25T23:49:02.635 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.647+0000 7f60e0945640 1 -- 172.21.15.57:0/669023308 wait complete. 2024-04-25T23:49:02.636 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.647+0000 7f60e0945640 1 Processor -- start 2024-04-25T23:49:02.636 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.647+0000 7f60e0945640 1 -- start start 2024-04-25T23:49:02.638 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.647+0000 7f60e0945640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60dc074b50 0x7f60dc0aab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:02.638 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.647+0000 7f60e0945640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60dc13dd40 0x7f60dc13ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:02.638 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.647+0000 7f60e0945640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f60dc0a9f90 0x7f60dc0aca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:02.638 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.647+0000 7f60e0945640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f60dc14ef50 con 0x7f60dc074b50 2024-04-25T23:49:02.638 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.647+0000 7f60e0945640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f60dc14edd0 con 0x7f60dc13dd40 2024-04-25T23:49:02.638 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.647+0000 7f60e0945640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f60dc14f0d0 con 0x7f60dc0a9f90 2024-04-25T23:49:02.638 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.647+0000 7f60dad9d640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f60dc0a9f90 0x7f60dc0aca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:02.638 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.647+0000 7f60da59c640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60dc074b50 0x7f60dc0aab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:02.638 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.647+0000 7f60dad9d640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f60dc0a9f90 0x7f60dc0aca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3301/0 says I am v2:172.21.15.57:47608/0 (socket says 172.21.15.57:47608) 2024-04-25T23:49:02.638 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.647+0000 7f60dad9d640 1 -- 172.21.15.57:0/515049131 learned_addr learned my addr 172.21.15.57:0/515049131 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:49:02.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.647+0000 7f60d9d9b640 1 --2- 172.21.15.57:0/515049131 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60dc13dd40 0x7f60dc13ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:02.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.647+0000 7f60dad9d640 1 -- 172.21.15.57:0/515049131 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60dc13dd40 msgr2=0x7f60dc13ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:02.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.647+0000 7f60dad9d640 1 --2- 172.21.15.57:0/515049131 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60dc13dd40 0x7f60dc13ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.647+0000 7f60dad9d640 1 -- 172.21.15.57:0/515049131 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60dc074b50 msgr2=0x7f60dc0aab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:02.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.647+0000 7f60dad9d640 1 --2- 172.21.15.57:0/515049131 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60dc074b50 0x7f60dc0aab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.647+0000 7f60dad9d640 1 -- 172.21.15.57:0/515049131 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60dc0ac0a0 con 0x7f60dc0a9f90 2024-04-25T23:49:02.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.647+0000 7f60d9d9b640 1 --2- 172.21.15.57:0/515049131 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60dc13dd40 0x7f60dc13ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T23:49:02.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.651+0000 7f60dad9d640 1 --2- 172.21.15.57:0/515049131 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f60dc0a9f90 0x7f60dc0aca10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f60d400c370 tx=0x7f60d400c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:02.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.651+0000 7f60d2ffd640 1 -- 172.21.15.57:0/515049131 <== mon.2 v2:172.21.15.57:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d4005220 con 0x7f60dc0a9f90 2024-04-25T23:49:02.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.651+0000 7f60d2ffd640 1 -- 172.21.15.57:0/515049131 <== mon.2 v2:172.21.15.57:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60d40053c0 con 0x7f60dc0a9f90 2024-04-25T23:49:02.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.651+0000 7f60e0945640 1 -- 172.21.15.57:0/515049131 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60dc0ac310 con 0x7f60dc0a9f90 2024-04-25T23:49:02.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.651+0000 7f60d2ffd640 1 -- 172.21.15.57:0/515049131 <== mon.2 v2:172.21.15.57:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d4007270 con 0x7f60dc0a9f90 2024-04-25T23:49:02.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.651+0000 7f60e0945640 1 -- 172.21.15.57:0/515049131 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f60dc1404f0 con 0x7f60dc0a9f90 2024-04-25T23:49:02.639 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.651+0000 7f60e0945640 1 -- 172.21.15.57:0/515049131 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60dc068210 con 0x7f60dc0a9f90 2024-04-25T23:49:02.640 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.651+0000 7f60d2ffd640 1 -- 172.21.15.57:0/515049131 <== mon.2 v2:172.21.15.57:3301/0 4 ==== mgrmap(e 5) ==== 87410+0+0 (secure 0 0 0) 0x7f60d4019020 con 0x7f60dc0a9f90 2024-04-25T23:49:02.640 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.651+0000 7f60d2ffd640 1 --2- 172.21.15.57:0/515049131 >> v2:172.21.15.80:6816/2493457677 conn(0x7f60bc067880 0x7f60bc069d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:02.640 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.651+0000 7f60d2ffd640 1 -- 172.21.15.57:0/515049131 <== mon.2 v2:172.21.15.57:3301/0 5 ==== osd_map(17..17 src has 1..17) ==== 4090+0+0 (secure 0 0 0) 0x7f60d4084770 con 0x7f60dc0a9f90 2024-04-25T23:49:02.640 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.651+0000 7f60da59c640 1 --2- 172.21.15.57:0/515049131 >> v2:172.21.15.80:6816/2493457677 conn(0x7f60bc067880 0x7f60bc069d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:02.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.651+0000 7f60da59c640 1 --2- 172.21.15.57:0/515049131 >> v2:172.21.15.80:6816/2493457677 conn(0x7f60bc067880 0x7f60bc069d40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f60cc0011f0 tx=0x7f60cc007d40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:02.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.655+0000 7f60d2ffd640 1 -- 172.21.15.57:0/515049131 <== mon.2 v2:172.21.15.57:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f60d4055900 con 0x7f60dc0a9f90 2024-04-25T23:49:02.796 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.807+0000 7f60e0945640 1 -- 172.21.15.57:0/515049131 --> v2:172.21.15.80:6816/2493457677 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f60dc06c740 con 0x7f60bc067880 2024-04-25T23:49:02.799 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.811+0000 7f60d2ffd640 1 -- 172.21.15.57:0/515049131 <== mgr.4100 v2:172.21.15.80:6816/2493457677 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+35618 (secure 0 0 0) 0x7f60dc06c740 con 0x7f60bc067880 2024-04-25T23:49:02.800 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-25T23:49:02.800 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2024-04-25T23:49:02.801 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.815+0000 7f60e0945640 1 -- 172.21.15.57:0/515049131 >> v2:172.21.15.80:6816/2493457677 conn(0x7f60bc067880 msgr2=0x7f60bc069d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:02.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.815+0000 7f60e0945640 1 --2- 172.21.15.57:0/515049131 >> v2:172.21.15.80:6816/2493457677 conn(0x7f60bc067880 0x7f60bc069d40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f60cc0011f0 tx=0x7f60cc007d40 comp rx=0 tx=0).stop 2024-04-25T23:49:02.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.815+0000 7f60e0945640 1 -- 172.21.15.57:0/515049131 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f60dc0a9f90 msgr2=0x7f60dc0aca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:02.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.815+0000 7f60e0945640 1 --2- 172.21.15.57:0/515049131 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f60dc0a9f90 0x7f60dc0aca10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f60d400c370 tx=0x7f60d400c830 comp rx=0 tx=0).stop 2024-04-25T23:49:02.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.815+0000 7f60e0945640 1 -- 172.21.15.57:0/515049131 shutdown_connections 2024-04-25T23:49:02.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.815+0000 7f60e0945640 1 --2- 172.21.15.57:0/515049131 >> v2:172.21.15.80:6816/2493457677 conn(0x7f60bc067880 0x7f60bc069d40 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.815+0000 7f60e0945640 1 --2- 172.21.15.57:0/515049131 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60dc13dd40 0x7f60dc13ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.815+0000 7f60e0945640 1 --2- 172.21.15.57:0/515049131 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f60dc0a9f90 0x7f60dc0aca10 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.815+0000 7f60e0945640 1 --2- 172.21.15.57:0/515049131 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60dc074b50 0x7f60dc0aab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.815+0000 7f60e0945640 1 -- 172.21.15.57:0/515049131 >> 172.21.15.57:0/515049131 conn(0x7f60dc0b5320 msgr2=0x7f60dc0b2b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:49:02.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.815+0000 7f60e0945640 1 -- 172.21.15.57:0/515049131 shutdown_connections 2024-04-25T23:49:02.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.815+0000 7f60e0945640 1 -- 172.21.15.57:0/515049131 wait complete. 2024-04-25T23:49:02.817 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":12,"stamp":"2024-04-25T23:49:00.823595+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":145,"num_write_kb":2154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":105,"ondisk_log_size":105,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":217444,"kb_used_data":1212,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754757276,"statfs":{"total":773094113280,"available":772871450624,"internally_reserved":0,"allocated":1241088,"data_stored":1216340,"data_compressed":9127,"data_compressed_allocated":53248,"data_compressed_original":839680,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-25T23:48:59.900430+0000","last_change":"2024-04-25T23:48:58.911464+0000","last_active":"2024-04-25T23:48:59.900430+0000","last_peered":"2024-04-25T23:48:59.900430+0000","last_clean":"2024-04-25T23:48:59.900430+0000","last_became_active":"2024-04-25T23:48:58.911101+0000","last_became_peered":"2024-04-25T23:48:58.911101+0000","last_unstale":"2024-04-25T23:48:59.900430+0000","last_undegraded":"2024-04-25T23:48:59.900430+0000","last_fullsized":"2024-04-25T23:48:59.900430+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_clean_scrub_stamp":"2024-04-25T23:48:57.875228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:06:28.527623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-25T23:48:59.908419+0000","last_change":"2024-04-25T23:48:58.906152+0000","last_active":"2024-04-25T23:48:59.908419+0000","last_peered":"2024-04-25T23:48:59.908419+0000","last_clean":"2024-04-25T23:48:59.908419+0000","last_became_active":"2024-04-25T23:48:58.905805+0000","last_became_peered":"2024-04-25T23:48:58.905805+0000","last_unstale":"2024-04-25T23:48:59.908419+0000","last_undegraded":"2024-04-25T23:48:59.908419+0000","last_fullsized":"2024-04-25T23:48:59.908419+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_clean_scrub_stamp":"2024-04-25T23:48:57.875228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:05:52.310323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-25T23:48:59.910913+0000","last_change":"2024-04-25T23:48:59.122975+0000","last_active":"2024-04-25T23:48:59.910913+0000","last_peered":"2024-04-25T23:48:59.910913+0000","last_clean":"2024-04-25T23:48:59.910913+0000","last_became_active":"2024-04-25T23:48:59.122193+0000","last_became_peered":"2024-04-25T23:48:59.122193+0000","last_unstale":"2024-04-25T23:48:59.910913+0000","last_undegraded":"2024-04-25T23:48:59.910913+0000","last_fullsized":"2024-04-25T23:48:59.910913+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_clean_scrub_stamp":"2024-04-25T23:48:57.875228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:09:25.578965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-25T23:48:59.908512+0000","last_change":"2024-04-25T23:48:59.120607+0000","last_active":"2024-04-25T23:48:59.908512+0000","last_peered":"2024-04-25T23:48:59.908512+0000","last_clean":"2024-04-25T23:48:59.908512+0000","last_became_active":"2024-04-25T23:48:59.119535+0000","last_became_peered":"2024-04-25T23:48:59.119535+0000","last_unstale":"2024-04-25T23:48:59.908512+0000","last_undegraded":"2024-04-25T23:48:59.908512+0000","last_fullsized":"2024-04-25T23:48:59.908512+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_clean_scrub_stamp":"2024-04-25T23:48:57.875228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:40:05.201442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-25T23:49:00.039861+0000","last_change":"2024-04-25T23:48:58.920072+0000","last_active":"2024-04-25T23:49:00.039861+0000","last_peered":"2024-04-25T23:49:00.039861+0000","last_clean":"2024-04-25T23:49:00.039861+0000","last_became_active":"2024-04-25T23:48:58.919554+0000","last_became_peered":"2024-04-25T23:48:58.919554+0000","last_unstale":"2024-04-25T23:49:00.039861+0000","last_undegraded":"2024-04-25T23:49:00.039861+0000","last_fullsized":"2024-04-25T23:49:00.039861+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_clean_scrub_stamp":"2024-04-25T23:48:57.875228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:47:42.230724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-25T23:49:00.036840+0000","last_change":"2024-04-25T23:48:58.910610+0000","last_active":"2024-04-25T23:49:00.036840+0000","last_peered":"2024-04-25T23:49:00.036840+0000","last_clean":"2024-04-25T23:49:00.036840+0000","last_became_active":"2024-04-25T23:48:58.910228+0000","last_became_peered":"2024-04-25T23:48:58.910228+0000","last_unstale":"2024-04-25T23:49:00.036840+0000","last_undegraded":"2024-04-25T23:49:00.036840+0000","last_fullsized":"2024-04-25T23:49:00.036840+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_clean_scrub_stamp":"2024-04-25T23:48:57.875228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:40:43.690806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-25T23:48:59.910916+0000","last_change":"2024-04-25T23:48:58.913466+0000","last_active":"2024-04-25T23:48:59.910916+0000","last_peered":"2024-04-25T23:48:59.910916+0000","last_clean":"2024-04-25T23:48:59.910916+0000","last_became_active":"2024-04-25T23:48:58.912966+0000","last_became_peered":"2024-04-25T23:48:58.912966+0000","last_unstale":"2024-04-25T23:48:59.910916+0000","last_undegraded":"2024-04-25T23:48:59.910916+0000","last_fullsized":"2024-04-25T23:48:59.910916+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_clean_scrub_stamp":"2024-04-25T23:48:57.875228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:25:56.738346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-25T23:49:00.039861+0000","last_change":"2024-04-25T23:48:58.915463+0000","last_active":"2024-04-25T23:49:00.039861+0000","last_peered":"2024-04-25T23:49:00.039861+0000","last_clean":"2024-04-25T23:49:00.039861+0000","last_became_active":"2024-04-25T23:48:58.914379+0000","last_became_peered":"2024-04-25T23:48:58.914379+0000","last_unstale":"2024-04-25T23:49:00.039861+0000","last_undegraded":"2024-04-25T23:49:00.039861+0000","last_fullsized":"2024-04-25T23:49:00.039861+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_clean_scrub_stamp":"2024-04-25T23:48:57.875228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:11:09.852719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'105","reported_seq":147,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-25T23:48:59.929895+0000","last_change":"2024-04-25T23:48:59.124853+0000","last_active":"2024-04-25T23:48:59.929895+0000","last_peered":"2024-04-25T23:48:59.929895+0000","last_clean":"2024-04-25T23:48:59.929895+0000","last_became_active":"2024-04-25T23:48:59.122379+0000","last_became_peered":"2024-04-25T23:48:59.122379+0000","last_unstale":"2024-04-25T23:48:59.929895+0000","last_undegraded":"2024-04-25T23:48:59.929895+0000","last_fullsized":"2024-04-25T23:48:59.929895+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_clean_scrub_stamp":"2024-04-25T23:48:57.875228+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:57:31.186906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":145,"num_write_kb":2154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":145,"num_write_kb":2154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":934976,"data_compressed":11912,"data_compressed_allocated":57344,"data_compressed_original":901120,"omap_allocated":0,"internal_metadata":0},"log_size":105,"ondisk_log_size":105,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542146,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27172,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":147456,"data_stored":45416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":147456,"data_stored":45416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542146,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27168,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":147456,"data_stored":45416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574850,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":196608,"data_stored":512904,"data_compressed":5956,"data_compressed_allocated":28672,"data_compressed_original":450560,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574850,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":139264,"data_stored":40418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574850,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":139264,"data_stored":40418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574850,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":139264,"data_stored":40418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574850,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":184320,"data_stored":445934,"data_compressed":3171,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":467488,"data_compressed":5956,"data_compressed_allocated":28672,"data_compressed_original":450560,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":467488,"data_compressed":5956,"data_compressed_allocated":28672,"data_compressed_original":450560,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T23:49:02.818 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T23:49:02.939 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.951+0000 7f6058a00640 1 Processor -- start 2024-04-25T23:49:02.939 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.951+0000 7f6058a00640 1 -- start start 2024-04-25T23:49:02.940 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.951+0000 7f6058a00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f60540b54f0 0x7f6054146010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:02.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.951+0000 7f6058a00640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f60540b2530 0x7f605414cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:02.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.951+0000 7f6058a00640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60540b2960 con 0x7f6054146550 2024-04-25T23:49:02.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.951+0000 7f6058a00640 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 -- 0x7f60540b58d0 con 0x7f60540b2530 2024-04-25T23:49:02.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.951+0000 7f6058a00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f6054145b80 con 0x7f60540b54f0 2024-04-25T23:49:02.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.951+0000 7f6052d9d640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f60540b2530 0x7f605414cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:02.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.951+0000 7f6052d9d640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f60540b2530 0x7f605414cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33688/0 (socket says 172.21.15.57:33688) 2024-04-25T23:49:02.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.951+0000 7f6052d9d640 1 -- 172.21.15.57:0/1755984982 learned_addr learned my addr 172.21.15.57:0/1755984982 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:49:02.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.951+0000 7f6052d9d640 1 -- 172.21.15.57:0/1755984982 >> v2:172.21.15.80:3300/0 conn(0x7f60540b54f0 msgr2=0x7f6054146010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:49:02.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.951+0000 7f6052d9d640 1 --2- 172.21.15.57:0/1755984982 >> v2:172.21.15.80:3300/0 conn(0x7f60540b54f0 0x7f6054146010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.951+0000 7f6052d9d640 1 -- 172.21.15.57:0/1755984982 >> v1:172.21.15.57:6789/0 conn(0x7f6054146550 legacy=0x7f60540b1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:02.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.951+0000 7f6052d9d640 1 -- 172.21.15.57:0/1755984982 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6054142600 con 0x7f60540b2530 2024-04-25T23:49:02.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.955+0000 7f6052d9d640 1 --2- 172.21.15.57:0/1755984982 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60540b2530 0x7f605414cf10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6054072d20 tx=0x7f604c01c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c019d5b3bfd5f5ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:02.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.955+0000 7f6051d9b640 1 -- 172.21.15.57:0/1755984982 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604c0a56c0 con 0x7f60540b2530 2024-04-25T23:49:02.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.955+0000 7f6051d9b640 1 -- 172.21.15.57:0/1755984982 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f604c0a5860 con 0x7f60540b2530 2024-04-25T23:49:02.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.955+0000 7f6051d9b640 1 -- 172.21.15.57:0/1755984982 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604c0a5c30 con 0x7f60540b2530 2024-04-25T23:49:02.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.955+0000 7f6058a00640 1 -- 172.21.15.57:0/1755984982 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60540b2530 msgr2=0x7f605414cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:02.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.955+0000 7f6058a00640 1 --2- 172.21.15.57:0/1755984982 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60540b2530 0x7f605414cf10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6054072d20 tx=0x7f604c01c860 comp rx=0 tx=0).stop 2024-04-25T23:49:02.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.955+0000 7f6058a00640 1 -- 172.21.15.57:0/1755984982 shutdown_connections 2024-04-25T23:49:02.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.955+0000 7f6058a00640 1 --2- 172.21.15.57:0/1755984982 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60540b2530 0x7f605414cf10 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.955+0000 7f6058a00640 1 --2- 172.21.15.57:0/1755984982 >> v2:172.21.15.80:3300/0 conn(0x7f60540b54f0 0x7f6054146010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.955+0000 7f6058a00640 1 -- 172.21.15.57:0/1755984982 >> 172.21.15.57:0/1755984982 conn(0x7f6054074cf0 msgr2=0x7f6054075100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:49:02.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.955+0000 7f6058a00640 1 -- 172.21.15.57:0/1755984982 shutdown_connections 2024-04-25T23:49:02.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.955+0000 7f6058a00640 1 -- 172.21.15.57:0/1755984982 wait complete. 2024-04-25T23:49:02.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.955+0000 7f6058a00640 1 Processor -- start 2024-04-25T23:49:02.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.955+0000 7f6058a00640 1 -- start start 2024-04-25T23:49:02.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.955+0000 7f6058a00640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f60540b2530 0x7f605413f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:02.946 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.955+0000 7f6058a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60540b54f0 0x7f60540b17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:02.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.955+0000 7f6058a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6054146550 0x7f60541d80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:02.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.955+0000 7f6058a00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f60541469c0 con 0x7f6054146550 2024-04-25T23:49:02.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.955+0000 7f6058a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f60540ab390 con 0x7f60540b54f0 2024-04-25T23:49:02.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.955+0000 7f6058a00640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f60540a9770 con 0x7f60540b2530 2024-04-25T23:49:02.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.959+0000 7f605259c640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f60540b2530 0x7f605413f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:02.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.959+0000 7f605259c640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f60540b2530 0x7f605413f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3301/0 says I am v2:172.21.15.57:47622/0 (socket says 172.21.15.57:47622) 2024-04-25T23:49:02.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.959+0000 7f605259c640 1 -- 172.21.15.57:0/3398476586 learned_addr learned my addr 172.21.15.57:0/3398476586 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:49:02.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.959+0000 7f6052d9d640 1 --2- 172.21.15.57:0/3398476586 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6054146550 0x7f60541d80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:02.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.959+0000 7f604bfff640 1 --2- 172.21.15.57:0/3398476586 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60540b54f0 0x7f60540b17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:02.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.959+0000 7f605259c640 1 -- 172.21.15.57:0/3398476586 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60540b54f0 msgr2=0x7f60540b17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:02.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.959+0000 7f605259c640 1 --2- 172.21.15.57:0/3398476586 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60540b54f0 0x7f60540b17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.959+0000 7f605259c640 1 -- 172.21.15.57:0/3398476586 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6054146550 msgr2=0x7f60541d80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:02.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.959+0000 7f605259c640 1 --2- 172.21.15.57:0/3398476586 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6054146550 0x7f60541d80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:02.947 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.959+0000 7f605259c640 1 -- 172.21.15.57:0/3398476586 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60540b2130 con 0x7f60540b2530 2024-04-25T23:49:02.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.959+0000 7f605259c640 1 --2- 172.21.15.57:0/3398476586 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f60540b2530 0x7f605413f310 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f604400b550 tx=0x7f604400ba10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:02.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.959+0000 7f604affd640 1 -- 172.21.15.57:0/3398476586 <== mon.2 v2:172.21.15.57:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6044014070 con 0x7f60540b2530 2024-04-25T23:49:02.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.959+0000 7f604affd640 1 -- 172.21.15.57:0/3398476586 <== mon.2 v2:172.21.15.57:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6044010610 con 0x7f60540b2530 2024-04-25T23:49:02.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.959+0000 7f6058a00640 1 -- 172.21.15.57:0/3398476586 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f605413fe10 con 0x7f60540b2530 2024-04-25T23:49:02.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.959+0000 7f604affd640 1 -- 172.21.15.57:0/3398476586 <== mon.2 v2:172.21.15.57:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60440109e0 con 0x7f60540b2530 2024-04-25T23:49:02.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.959+0000 7f6058a00640 1 -- 172.21.15.57:0/3398476586 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f60541d8620 con 0x7f60540b2530 2024-04-25T23:49:02.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.959+0000 7f6058a00640 1 -- 172.21.15.57:0/3398476586 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6054068a10 con 0x7f60540b2530 2024-04-25T23:49:02.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.959+0000 7f604affd640 1 -- 172.21.15.57:0/3398476586 <== mon.2 v2:172.21.15.57:3301/0 4 ==== mgrmap(e 5) ==== 87410+0+0 (secure 0 0 0) 0x7f6044017070 con 0x7f60540b2530 2024-04-25T23:49:02.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.959+0000 7f604affd640 1 --2- 172.21.15.57:0/3398476586 >> v2:172.21.15.80:6816/2493457677 conn(0x7f6034067880 0x7f6034069d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:02.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.959+0000 7f604affd640 1 -- 172.21.15.57:0/3398476586 <== mon.2 v2:172.21.15.57:3301/0 5 ==== osd_map(17..17 src has 1..17) ==== 4090+0+0 (secure 0 0 0) 0x7f6044084f90 con 0x7f60540b2530 2024-04-25T23:49:02.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.959+0000 7f604bfff640 1 --2- 172.21.15.57:0/3398476586 >> v2:172.21.15.80:6816/2493457677 conn(0x7f6034067880 0x7f6034069d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:02.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.959+0000 7f604bfff640 1 --2- 172.21.15.57:0/3398476586 >> v2:172.21.15.80:6816/2493457677 conn(0x7f6034067880 0x7f6034069d40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f60540b69b0 tx=0x7f603c00c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:02.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:02.963+0000 7f604affd640 1 -- 172.21.15.57:0/3398476586 <== mon.2 v2:172.21.15.57:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6044055ad0 con 0x7f60540b2530 2024-04-25T23:49:03.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.115+0000 7f6058a00640 1 -- 172.21.15.57:0/3398476586 --> v2:172.21.15.80:6816/2493457677 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f60540b4170 con 0x7f6034067880 2024-04-25T23:49:03.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.119+0000 7f604affd640 1 -- 172.21.15.57:0/3398476586 <== mgr.4100 v2:172.21.15.80:6816/2493457677 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+35596 (secure 0 0 0) 0x7f60540b4170 con 0x7f6034067880 2024-04-25T23:49:03.107 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-25T23:49:03.108 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2024-04-25T23:49:03.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.119+0000 7f6058a00640 1 -- 172.21.15.57:0/3398476586 >> v2:172.21.15.80:6816/2493457677 conn(0x7f6034067880 msgr2=0x7f6034069d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:03.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.119+0000 7f6058a00640 1 --2- 172.21.15.57:0/3398476586 >> v2:172.21.15.80:6816/2493457677 conn(0x7f6034067880 0x7f6034069d40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f60540b69b0 tx=0x7f603c00c040 comp rx=0 tx=0).stop 2024-04-25T23:49:03.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.119+0000 7f6058a00640 1 -- 172.21.15.57:0/3398476586 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f60540b2530 msgr2=0x7f605413f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:03.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.119+0000 7f6058a00640 1 --2- 172.21.15.57:0/3398476586 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f60540b2530 0x7f605413f310 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f604400b550 tx=0x7f604400ba10 comp rx=0 tx=0).stop 2024-04-25T23:49:03.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.119+0000 7f6058a00640 1 -- 172.21.15.57:0/3398476586 shutdown_connections 2024-04-25T23:49:03.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.119+0000 7f6058a00640 1 --2- 172.21.15.57:0/3398476586 >> v2:172.21.15.80:6816/2493457677 conn(0x7f6034067880 0x7f6034069d40 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:03.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.119+0000 7f6058a00640 1 --2- 172.21.15.57:0/3398476586 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60540b54f0 0x7f60540b17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:03.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.119+0000 7f6058a00640 1 --2- 172.21.15.57:0/3398476586 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6054146550 0x7f60541d80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:03.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.119+0000 7f6058a00640 1 --2- 172.21.15.57:0/3398476586 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f60540b2530 0x7f605413f310 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:03.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.119+0000 7f6058a00640 1 -- 172.21.15.57:0/3398476586 >> 172.21.15.57:0/3398476586 conn(0x7f6054074cf0 msgr2=0x7f6054141070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:49:03.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.123+0000 7f6058a00640 1 -- 172.21.15.57:0/3398476586 shutdown_connections 2024-04-25T23:49:03.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.123+0000 7f6058a00640 1 -- 172.21.15.57:0/3398476586 wait complete. 2024-04-25T23:49:03.124 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2024-04-25T23:49:02.824230+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":12,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":217488,"kb_used_data":1256,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754757232,"statfs":{"total":773094113280,"available":772871405568,"internally_reserved":0,"allocated":1286144,"data_stored":1421184,"data_compressed":12118,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":14,"num_write_kb":262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000634"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-25T23:48:59.900430+0000","last_change":"2024-04-25T23:48:58.911464+0000","last_active":"2024-04-25T23:48:59.900430+0000","last_peered":"2024-04-25T23:48:59.900430+0000","last_clean":"2024-04-25T23:48:59.900430+0000","last_became_active":"2024-04-25T23:48:58.911101+0000","last_became_peered":"2024-04-25T23:48:58.911101+0000","last_unstale":"2024-04-25T23:48:59.900430+0000","last_undegraded":"2024-04-25T23:48:59.900430+0000","last_fullsized":"2024-04-25T23:48:59.900430+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_clean_scrub_stamp":"2024-04-25T23:48:57.875228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:06:28.527623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-25T23:48:59.908419+0000","last_change":"2024-04-25T23:48:58.906152+0000","last_active":"2024-04-25T23:48:59.908419+0000","last_peered":"2024-04-25T23:48:59.908419+0000","last_clean":"2024-04-25T23:48:59.908419+0000","last_became_active":"2024-04-25T23:48:58.905805+0000","last_became_peered":"2024-04-25T23:48:58.905805+0000","last_unstale":"2024-04-25T23:48:59.908419+0000","last_undegraded":"2024-04-25T23:48:59.908419+0000","last_fullsized":"2024-04-25T23:48:59.908419+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_clean_scrub_stamp":"2024-04-25T23:48:57.875228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:05:52.310323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-25T23:48:59.910913+0000","last_change":"2024-04-25T23:48:59.122975+0000","last_active":"2024-04-25T23:48:59.910913+0000","last_peered":"2024-04-25T23:48:59.910913+0000","last_clean":"2024-04-25T23:48:59.910913+0000","last_became_active":"2024-04-25T23:48:59.122193+0000","last_became_peered":"2024-04-25T23:48:59.122193+0000","last_unstale":"2024-04-25T23:48:59.910913+0000","last_undegraded":"2024-04-25T23:48:59.910913+0000","last_fullsized":"2024-04-25T23:48:59.910913+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_clean_scrub_stamp":"2024-04-25T23:48:57.875228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:09:25.578965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-25T23:48:59.908512+0000","last_change":"2024-04-25T23:48:59.120607+0000","last_active":"2024-04-25T23:48:59.908512+0000","last_peered":"2024-04-25T23:48:59.908512+0000","last_clean":"2024-04-25T23:48:59.908512+0000","last_became_active":"2024-04-25T23:48:59.119535+0000","last_became_peered":"2024-04-25T23:48:59.119535+0000","last_unstale":"2024-04-25T23:48:59.908512+0000","last_undegraded":"2024-04-25T23:48:59.908512+0000","last_fullsized":"2024-04-25T23:48:59.908512+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_clean_scrub_stamp":"2024-04-25T23:48:57.875228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:40:05.201442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-25T23:49:00.039861+0000","last_change":"2024-04-25T23:48:58.920072+0000","last_active":"2024-04-25T23:49:00.039861+0000","last_peered":"2024-04-25T23:49:00.039861+0000","last_clean":"2024-04-25T23:49:00.039861+0000","last_became_active":"2024-04-25T23:48:58.919554+0000","last_became_peered":"2024-04-25T23:48:58.919554+0000","last_unstale":"2024-04-25T23:49:00.039861+0000","last_undegraded":"2024-04-25T23:49:00.039861+0000","last_fullsized":"2024-04-25T23:49:00.039861+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_clean_scrub_stamp":"2024-04-25T23:48:57.875228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:47:42.230724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-25T23:49:00.036840+0000","last_change":"2024-04-25T23:48:58.910610+0000","last_active":"2024-04-25T23:49:00.036840+0000","last_peered":"2024-04-25T23:49:00.036840+0000","last_clean":"2024-04-25T23:49:00.036840+0000","last_became_active":"2024-04-25T23:48:58.910228+0000","last_became_peered":"2024-04-25T23:48:58.910228+0000","last_unstale":"2024-04-25T23:49:00.036840+0000","last_undegraded":"2024-04-25T23:49:00.036840+0000","last_fullsized":"2024-04-25T23:49:00.036840+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_clean_scrub_stamp":"2024-04-25T23:48:57.875228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:40:43.690806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-25T23:48:59.910916+0000","last_change":"2024-04-25T23:48:58.913466+0000","last_active":"2024-04-25T23:48:59.910916+0000","last_peered":"2024-04-25T23:48:59.910916+0000","last_clean":"2024-04-25T23:48:59.910916+0000","last_became_active":"2024-04-25T23:48:58.912966+0000","last_became_peered":"2024-04-25T23:48:58.912966+0000","last_unstale":"2024-04-25T23:48:59.910916+0000","last_undegraded":"2024-04-25T23:48:59.910916+0000","last_fullsized":"2024-04-25T23:48:59.910916+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_clean_scrub_stamp":"2024-04-25T23:48:57.875228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:25:56.738346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-25T23:49:00.039861+0000","last_change":"2024-04-25T23:48:58.915463+0000","last_active":"2024-04-25T23:49:00.039861+0000","last_peered":"2024-04-25T23:49:00.039861+0000","last_clean":"2024-04-25T23:49:00.039861+0000","last_became_active":"2024-04-25T23:48:58.914379+0000","last_became_peered":"2024-04-25T23:48:58.914379+0000","last_unstale":"2024-04-25T23:49:00.039861+0000","last_undegraded":"2024-04-25T23:49:00.039861+0000","last_fullsized":"2024-04-25T23:49:00.039861+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_clean_scrub_stamp":"2024-04-25T23:48:57.875228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:11:09.852719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":162,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-25T23:49:00.710303+0000","last_change":"2024-04-25T23:48:59.124853+0000","last_active":"2024-04-25T23:49:00.710303+0000","last_peered":"2024-04-25T23:49:00.710303+0000","last_clean":"2024-04-25T23:49:00.710303+0000","last_became_active":"2024-04-25T23:48:59.122379+0000","last_became_peered":"2024-04-25T23:48:59.122379+0000","last_unstale":"2024-04-25T23:49:00.710303+0000","last_undegraded":"2024-04-25T23:49:00.710303+0000","last_fullsized":"2024-04-25T23:49:00.710303+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_clean_scrub_stamp":"2024-04-25T23:48:57.875228+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:57:31.186906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":12118,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542146,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27172,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":147456,"data_stored":45416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":147456,"data_stored":45416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542146,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27168,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":147456,"data_stored":45416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":200704,"data_stored":574344,"data_compressed":6059,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":147456,"data_stored":45416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":147456,"data_stored":45416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":147456,"data_stored":45416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":200704,"data_stored":574344,"data_compressed":6059,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":6059,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":6059,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T23:49:03.125 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T23:49:03.125 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T23:49:03.126 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-25T23:49:03.126 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T23:49:03.244 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.255+0000 7f2993511640 1 Processor -- start 2024-04-25T23:49:03.244 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.255+0000 7f2993511640 1 -- start start 2024-04-25T23:49:03.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.255+0000 7f2993511640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f298c0b1290 0x7f298c139e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:03.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.255+0000 7f2993511640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f298c0a9570 con 0x7f298c0b1670 2024-04-25T23:49:03.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.255+0000 7f2993511640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f298c0723e0 con 0x7f298c06f490 2024-04-25T23:49:03.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.255+0000 7f2993511640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f298c073540 con 0x7f298c0b1290 2024-04-25T23:49:03.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.255+0000 7f29912ad640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f298c0b1290 0x7f298c139e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:03.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.255+0000 7f2990aac640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f298c0b1670 0x7f298c140340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:49714/0 (socket says 172.21.15.57:49714) 2024-04-25T23:49:03.245 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.255+0000 7f2990aac640 1 -- 172.21.15.57:0/1996107808 learned_addr learned my addr 172.21.15.57:0/1996107808 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-04-25T23:49:03.246 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.259+0000 7f29912ad640 1 -- 172.21.15.57:0/1996107808 >> v1:172.21.15.80:6789/0 conn(0x7f298c06f490 legacy=0x7f298c14ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:49:03.246 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.259+0000 7f29912ad640 1 -- 172.21.15.57:0/1996107808 >> v1:172.21.15.57:6789/0 conn(0x7f298c0b1670 legacy=0x7f298c140340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:03.246 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.259+0000 7f2983fff640 1 -- 172.21.15.57:0/1996107808 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2712823919 0 0) 0x7f298c0a9570 con 0x7f298c0b1670 2024-04-25T23:49:03.246 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.259+0000 7f29912ad640 1 -- 172.21.15.57:0/1996107808 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f298c140f90 con 0x7f298c0b1290 2024-04-25T23:49:03.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.259+0000 7f29912ad640 1 --2- 172.21.15.57:0/1996107808 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f298c0b1290 0x7f298c139e40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f298c0736c0 tx=0x7f297c01c340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c750a6f662c3c3b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:03.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.259+0000 7f2983fff640 1 -- 172.21.15.57:0/1996107808 <== mon.2 v2:172.21.15.57:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f297c0b2070 con 0x7f298c0b1290 2024-04-25T23:49:03.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.259+0000 7f2983fff640 1 -- 172.21.15.57:0/1996107808 <== mon.2 v2:172.21.15.57:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f297c0a9070 con 0x7f298c0b1290 2024-04-25T23:49:03.247 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.259+0000 7f2983fff640 1 -- 172.21.15.57:0/1996107808 <== mon.2 v2:172.21.15.57:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f297c0ad880 con 0x7f298c0b1290 2024-04-25T23:49:03.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.259+0000 7f2993511640 1 -- 172.21.15.57:0/1996107808 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f298c0b1290 msgr2=0x7f298c139e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:03.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.259+0000 7f2993511640 1 --2- 172.21.15.57:0/1996107808 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f298c0b1290 0x7f298c139e40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f298c0736c0 tx=0x7f297c01c340 comp rx=0 tx=0).stop 2024-04-25T23:49:03.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.259+0000 7f2993511640 1 -- 172.21.15.57:0/1996107808 shutdown_connections 2024-04-25T23:49:03.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.259+0000 7f2993511640 1 --2- 172.21.15.57:0/1996107808 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f298c0b1290 0x7f298c139e40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:03.248 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.259+0000 7f2993511640 1 -- 172.21.15.57:0/1996107808 >> 172.21.15.57:0/1996107808 conn(0x7f298c0b4740 msgr2=0x7f298c0b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:49:03.249 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.259+0000 7f2993511640 1 -- 172.21.15.57:0/1996107808 shutdown_connections 2024-04-25T23:49:03.249 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.259+0000 7f2993511640 1 -- 172.21.15.57:0/1996107808 wait complete. 2024-04-25T23:49:03.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.259+0000 7f2993511640 1 Processor -- start 2024-04-25T23:49:03.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.263+0000 7f2993511640 1 -- start start 2024-04-25T23:49:03.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.263+0000 7f2993511640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f298c06f490 0x7f298c13d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:03.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.263+0000 7f2993511640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f298c0b1290 0x7f298c136350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:03.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.263+0000 7f2993511640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f298c13d6b0 0x7f298c136890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:03.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.263+0000 7f2993511640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f298c14ef60 con 0x7f298c0b1290 2024-04-25T23:49:03.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.263+0000 7f2993511640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f298c14ede0 con 0x7f298c13d6b0 2024-04-25T23:49:03.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.263+0000 7f2993511640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f298c14f0e0 con 0x7f298c06f490 2024-04-25T23:49:03.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.263+0000 7f2990aac640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f298c0b1290 0x7f298c136350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:03.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.263+0000 7f29912ad640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f298c06f490 0x7f298c13d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:03.251 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.263+0000 7f29912ad640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f298c06f490 0x7f298c13d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3301/0 says I am v2:172.21.15.57:47636/0 (socket says 172.21.15.57:47636) 2024-04-25T23:49:03.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.263+0000 7f29912ad640 1 -- 172.21.15.57:0/2288748766 learned_addr learned my addr 172.21.15.57:0/2288748766 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:49:03.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.263+0000 7f2990aac640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f298c0b1290 0x7f298c136350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33696/0 (socket says 172.21.15.57:33696) 2024-04-25T23:49:03.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.263+0000 7f29912ad640 1 -- 172.21.15.57:0/2288748766 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f298c13d6b0 msgr2=0x7f298c136890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:03.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.263+0000 7f29912ad640 1 --2- 172.21.15.57:0/2288748766 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f298c13d6b0 0x7f298c136890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:03.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.263+0000 7f29912ad640 1 -- 172.21.15.57:0/2288748766 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f298c0b1290 msgr2=0x7f298c136350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:03.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.263+0000 7f29912ad640 1 --2- 172.21.15.57:0/2288748766 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f298c0b1290 0x7f298c136350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:03.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.263+0000 7f29912ad640 1 -- 172.21.15.57:0/2288748766 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f298c136dd0 con 0x7f298c06f490 2024-04-25T23:49:03.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.263+0000 7f2990aac640 1 --2- 172.21.15.57:0/2288748766 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f298c0b1290 0x7f298c136350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T23:49:03.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.263+0000 7f29912ad640 1 --2- 172.21.15.57:0/2288748766 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f298c06f490 0x7f298c13d170 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f297c0b0040 tx=0x7f297c01ab80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:03.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.263+0000 7f2981ffb640 1 -- 172.21.15.57:0/2288748766 <== mon.2 v2:172.21.15.57:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f297c0b2040 con 0x7f298c06f490 2024-04-25T23:49:03.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.263+0000 7f2981ffb640 1 -- 172.21.15.57:0/2288748766 <== mon.2 v2:172.21.15.57:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f297c0add80 con 0x7f298c06f490 2024-04-25T23:49:03.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.263+0000 7f2993511640 1 -- 172.21.15.57:0/2288748766 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f298c13c9f0 con 0x7f298c06f490 2024-04-25T23:49:03.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.263+0000 7f2981ffb640 1 -- 172.21.15.57:0/2288748766 <== mon.2 v2:172.21.15.57:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f297c004250 con 0x7f298c06f490 2024-04-25T23:49:03.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.263+0000 7f2993511640 1 -- 172.21.15.57:0/2288748766 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f298c1dea00 con 0x7f298c06f490 2024-04-25T23:49:03.252 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.263+0000 7f2993511640 1 -- 172.21.15.57:0/2288748766 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f298c068a10 con 0x7f298c06f490 2024-04-25T23:49:03.253 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.263+0000 7f2981ffb640 1 -- 172.21.15.57:0/2288748766 <== mon.2 v2:172.21.15.57:3301/0 4 ==== mgrmap(e 5) ==== 87410+0+0 (secure 0 0 0) 0x7f297c0ab070 con 0x7f298c06f490 2024-04-25T23:49:03.253 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.263+0000 7f2981ffb640 1 --2- 172.21.15.57:0/2288748766 >> v2:172.21.15.80:6816/2493457677 conn(0x7f29700678b0 0x7f2970069d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:03.253 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.263+0000 7f2981ffb640 1 -- 172.21.15.57:0/2288748766 <== mon.2 v2:172.21.15.57:3301/0 5 ==== osd_map(17..17 src has 1..17) ==== 4090+0+0 (secure 0 0 0) 0x7f297c11fff0 con 0x7f298c06f490 2024-04-25T23:49:03.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.263+0000 7f2990aac640 1 --2- 172.21.15.57:0/2288748766 >> v2:172.21.15.80:6816/2493457677 conn(0x7f29700678b0 0x7f2970069d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:03.254 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.267+0000 7f2990aac640 1 --2- 172.21.15.57:0/2288748766 >> v2:172.21.15.80:6816/2493457677 conn(0x7f29700678b0 0x7f2970069d70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f298c14b8c0 tx=0x7f2988007d60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:03.256 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.267+0000 7f2981ffb640 1 -- 172.21.15.57:0/2288748766 <== mon.2 v2:172.21.15.57:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f297c016050 con 0x7f298c06f490 2024-04-25T23:49:03.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.459+0000 7f2993511640 1 -- 172.21.15.57:0/2288748766 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f298c13f0a0 con 0x7f298c06f490 2024-04-25T23:49:03.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.459+0000 7f2981ffb640 1 -- 172.21.15.57:0/2288748766 <== mon.2 v2:172.21.15.57:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f297c0f1330 con 0x7f298c06f490 2024-04-25T23:49:03.446 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-25T23:49:03.446 INFO:teuthology.orchestra.run.smithi057.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T23:49:03.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.463+0000 7f2993511640 1 -- 172.21.15.57:0/2288748766 >> v2:172.21.15.80:6816/2493457677 conn(0x7f29700678b0 msgr2=0x7f2970069d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:03.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.463+0000 7f2993511640 1 --2- 172.21.15.57:0/2288748766 >> v2:172.21.15.80:6816/2493457677 conn(0x7f29700678b0 0x7f2970069d70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f298c14b8c0 tx=0x7f2988007d60 comp rx=0 tx=0).stop 2024-04-25T23:49:03.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.463+0000 7f2993511640 1 -- 172.21.15.57:0/2288748766 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f298c06f490 msgr2=0x7f298c13d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:03.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.463+0000 7f2993511640 1 --2- 172.21.15.57:0/2288748766 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f298c06f490 0x7f298c13d170 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f297c0b0040 tx=0x7f297c01ab80 comp rx=0 tx=0).stop 2024-04-25T23:49:03.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.463+0000 7f2993511640 1 -- 172.21.15.57:0/2288748766 shutdown_connections 2024-04-25T23:49:03.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.463+0000 7f2993511640 1 --2- 172.21.15.57:0/2288748766 >> v2:172.21.15.80:6816/2493457677 conn(0x7f29700678b0 0x7f2970069d70 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:03.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.463+0000 7f2993511640 1 --2- 172.21.15.57:0/2288748766 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f298c13d6b0 0x7f298c136890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:03.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.463+0000 7f2993511640 1 --2- 172.21.15.57:0/2288748766 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f298c0b1290 0x7f298c136350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:03.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.463+0000 7f2993511640 1 --2- 172.21.15.57:0/2288748766 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f298c06f490 0x7f298c13d170 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:03.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.463+0000 7f2993511640 1 -- 172.21.15.57:0/2288748766 >> 172.21.15.57:0/2288748766 conn(0x7f298c0b4740 msgr2=0x7f298c0b0340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:49:03.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.463+0000 7f2993511640 1 -- 172.21.15.57:0/2288748766 shutdown_connections 2024-04-25T23:49:03.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.463+0000 7f2993511640 1 -- 172.21.15.57:0/2288748766 wait complete. 2024-04-25T23:49:03.464 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-25T23:49:03.465 INFO:teuthology.run_tasks:Running task exec... 2024-04-25T23:49:03.475 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T23:49:03.475 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi057.front.sepia.ceph.com 2024-04-25T23:49:03.475 DEBUG:teuthology.orchestra.run.smithi057:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph osd pool create cephsqlite' 2024-04-25T23:49:03.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.607+0000 7f47f8535640 1 Processor -- start 2024-04-25T23:49:03.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.607+0000 7f47f8535640 1 -- start start 2024-04-25T23:49:03.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.607+0000 7f47f8535640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f47f00b5310 0x7f47f0142610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:03.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.607+0000 7f47f8535640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f47f00a9a70 0x7f47f0142b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:03.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.607+0000 7f47f8535640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47f00ab440 con 0x7f47f013f7c0 2024-04-25T23:49:03.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.607+0000 7f47f8535640 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 -- 0x7f47f00b56f0 con 0x7f47f00b5310 2024-04-25T23:49:03.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.607+0000 7f47f8535640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f47f0143090 con 0x7f47f00a9a70 2024-04-25T23:49:03.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.607+0000 7f47f6ad2640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f47f013f7c0 0x7f47f013fc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:49730/0 (socket says 172.21.15.57:49730) 2024-04-25T23:49:03.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.607+0000 7f47f62d1640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f47f00b5310 0x7f47f0142610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:03.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.607+0000 7f47f6ad2640 1 -- 172.21.15.57:0/2890540988 learned_addr learned my addr 172.21.15.57:0/2890540988 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-04-25T23:49:03.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.607+0000 7f47f5ad0640 1 --2- 172.21.15.57:0/2890540988 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f47f00a9a70 0x7f47f0142b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:03.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.607+0000 7f47f62d1640 1 -- 172.21.15.57:0/2890540988 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f47f00a9a70 msgr2=0x7f47f0142b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:49:03.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.607+0000 7f47f62d1640 1 --2- 172.21.15.57:0/2890540988 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f47f00a9a70 0x7f47f0142b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:03.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.607+0000 7f47f62d1640 1 -- 172.21.15.57:0/2890540988 >> v1:172.21.15.57:6789/0 conn(0x7f47f013f7c0 legacy=0x7f47f013fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:03.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.607+0000 7f47f52cf640 1 -- 172.21.15.57:0/2890540988 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1787010394 0 0) 0x7f47f00ab440 con 0x7f47f013f7c0 2024-04-25T23:49:03.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.607+0000 7f47f62d1640 1 -- 172.21.15.57:0/2890540988 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47f00b4920 con 0x7f47f00b5310 2024-04-25T23:49:03.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.611+0000 7f47f62d1640 1 --2- 172.21.15.57:0/2890540988 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f47f00b5310 0x7f47f0142610 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f47ec0097f0 tx=0x7f47ec01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20886bf036aba80b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:03.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.611+0000 7f47f52cf640 1 -- 172.21.15.57:0/2890540988 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ec0b1070 con 0x7f47f00b5310 2024-04-25T23:49:03.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.611+0000 7f47f52cf640 1 -- 172.21.15.57:0/2890540988 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47ec0ac620 con 0x7f47f00b5310 2024-04-25T23:49:03.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.611+0000 7f47f52cf640 1 -- 172.21.15.57:0/2890540988 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ec0ac9f0 con 0x7f47f00b5310 2024-04-25T23:49:03.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.615+0000 7f47f8535640 1 -- 172.21.15.57:0/2890540988 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f47f00b5310 msgr2=0x7f47f0142610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:03.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.615+0000 7f47f8535640 1 --2- 172.21.15.57:0/2890540988 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f47f00b5310 0x7f47f0142610 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f47ec0097f0 tx=0x7f47ec01c3c0 comp rx=0 tx=0).stop 2024-04-25T23:49:03.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.615+0000 7f47f8535640 1 -- 172.21.15.57:0/2890540988 shutdown_connections 2024-04-25T23:49:03.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.615+0000 7f47f8535640 1 --2- 172.21.15.57:0/2890540988 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f47f00a9a70 0x7f47f0142b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:03.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.615+0000 7f47f8535640 1 --2- 172.21.15.57:0/2890540988 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f47f00b5310 0x7f47f0142610 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:03.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.615+0000 7f47f8535640 1 -- 172.21.15.57:0/2890540988 >> 172.21.15.57:0/2890540988 conn(0x7f47f0067390 msgr2=0x7f47f00677a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:49:03.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.615+0000 7f47f8535640 1 -- 172.21.15.57:0/2890540988 shutdown_connections 2024-04-25T23:49:03.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.615+0000 7f47f8535640 1 -- 172.21.15.57:0/2890540988 wait complete. 2024-04-25T23:49:03.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.615+0000 7f47f8535640 1 Processor -- start 2024-04-25T23:49:03.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.615+0000 7f47f8535640 1 -- start start 2024-04-25T23:49:03.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.615+0000 7f47f8535640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f47f00a9a70 0x7f47f0141f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:03.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.615+0000 7f47f8535640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f47f00b5310 0x7f47f013b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:03.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.615+0000 7f47f8535640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47f013f7c0 0x7f47f013b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:03.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.615+0000 7f47f8535640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f47f014f860 con 0x7f47f00b5310 2024-04-25T23:49:03.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.615+0000 7f47f8535640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f47f014f6e0 con 0x7f47f013f7c0 2024-04-25T23:49:03.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.615+0000 7f47f8535640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f47f014f9e0 con 0x7f47f00a9a70 2024-04-25T23:49:03.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.615+0000 7f47f5ad0640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f47f00b5310 0x7f47f013b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:03.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.615+0000 7f47f62d1640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f47f00a9a70 0x7f47f0141f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:03.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.615+0000 7f47f5ad0640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f47f00b5310 0x7f47f013b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33726/0 (socket says 172.21.15.57:33726) 2024-04-25T23:49:03.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.615+0000 7f47f62d1640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f47f00a9a70 0x7f47f0141f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3301/0 says I am v2:172.21.15.57:47646/0 (socket says 172.21.15.57:47646) 2024-04-25T23:49:03.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.615+0000 7f47f5ad0640 1 -- 172.21.15.57:0/3244519441 learned_addr learned my addr 172.21.15.57:0/3244519441 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:49:03.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.615+0000 7f47f5ad0640 1 -- 172.21.15.57:0/3244519441 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f47f00a9a70 msgr2=0x7f47f0141f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:03.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.615+0000 7f47f5ad0640 1 --2- 172.21.15.57:0/3244519441 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f47f00a9a70 0x7f47f0141f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:03.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.615+0000 7f47f5ad0640 1 -- 172.21.15.57:0/3244519441 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47f013f7c0 msgr2=0x7f47f013b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:03.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.615+0000 7f47f5ad0640 1 --2- 172.21.15.57:0/3244519441 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47f013f7c0 0x7f47f013b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:03.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.615+0000 7f47f5ad0640 1 -- 172.21.15.57:0/3244519441 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47f013be00 con 0x7f47f00b5310 2024-04-25T23:49:03.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.615+0000 7f47f5ad0640 1 --2- 172.21.15.57:0/3244519441 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f47f00b5310 0x7f47f013b380 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f47e000e930 tx=0x7f47e000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:03.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.615+0000 7f47e6ffd640 1 -- 172.21.15.57:0/3244519441 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e0017070 con 0x7f47f00b5310 2024-04-25T23:49:03.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.615+0000 7f47e6ffd640 1 -- 172.21.15.57:0/3244519441 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47e000ca00 con 0x7f47f00b5310 2024-04-25T23:49:03.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.615+0000 7f47f8535640 1 -- 172.21.15.57:0/3244519441 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47f013a9a0 con 0x7f47f00b5310 2024-04-25T23:49:03.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.615+0000 7f47e6ffd640 1 -- 172.21.15.57:0/3244519441 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e000cdb0 con 0x7f47f00b5310 2024-04-25T23:49:03.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.615+0000 7f47f8535640 1 -- 172.21.15.57:0/3244519441 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47f013af60 con 0x7f47f00b5310 2024-04-25T23:49:03.606 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.615+0000 7f47f8535640 1 -- 172.21.15.57:0/3244519441 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47f0141660 con 0x7f47f00b5310 2024-04-25T23:49:03.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.619+0000 7f47e6ffd640 1 -- 172.21.15.57:0/3244519441 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) ==== 87410+0+0 (secure 0 0 0) 0x7f47e001a070 con 0x7f47f00b5310 2024-04-25T23:49:03.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.619+0000 7f47e6ffd640 1 --2- 172.21.15.57:0/3244519441 >> v2:172.21.15.80:6816/2493457677 conn(0x7f47d40678b0 0x7f47d4069d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:03.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.619+0000 7f47e6ffd640 1 -- 172.21.15.57:0/3244519441 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4090+0+0 (secure 0 0 0) 0x7f47e0085560 con 0x7f47f00b5310 2024-04-25T23:49:03.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.619+0000 7f47f62d1640 1 --2- 172.21.15.57:0/3244519441 >> v2:172.21.15.80:6816/2493457677 conn(0x7f47d40678b0 0x7f47d4069d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:03.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.619+0000 7f47f62d1640 1 --2- 172.21.15.57:0/3244519441 >> v2:172.21.15.80:6816/2493457677 conn(0x7f47d40678b0 0x7f47d4069d70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f47ec0ae000 tx=0x7f47ec0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:03.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.623+0000 7f47e6ffd640 1 -- 172.21.15.57:0/3244519441 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f47e00561f0 con 0x7f47f00b5310 2024-04-25T23:49:03.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.779+0000 7f47f8535640 1 -- 172.21.15.57:0/3244519441 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephsqlite"} v 0) -- 0x7f47f01434e0 con 0x7f47f00b5310 2024-04-25T23:49:03.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.939+0000 7f47e6ffd640 1 -- 172.21.15.57:0/3244519441 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephsqlite"}]=0 pool 'cephsqlite' created v18) ==== 110+0+0 (secure 0 0 0) 0x7f47e005a4f0 con 0x7f47f00b5310 2024-04-25T23:49:03.928 INFO:teuthology.orchestra.run.smithi057.stderr:pool 'cephsqlite' created 2024-04-25T23:49:03.930 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.943+0000 7f47f8535640 1 -- 172.21.15.57:0/3244519441 >> v2:172.21.15.80:6816/2493457677 conn(0x7f47d40678b0 msgr2=0x7f47d4069d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:03.930 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.943+0000 7f47f8535640 1 --2- 172.21.15.57:0/3244519441 >> v2:172.21.15.80:6816/2493457677 conn(0x7f47d40678b0 0x7f47d4069d70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f47ec0ae000 tx=0x7f47ec0a7000 comp rx=0 tx=0).stop 2024-04-25T23:49:03.930 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.943+0000 7f47f8535640 1 -- 172.21.15.57:0/3244519441 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f47f00b5310 msgr2=0x7f47f013b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:03.930 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.943+0000 7f47f8535640 1 --2- 172.21.15.57:0/3244519441 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f47f00b5310 0x7f47f013b380 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f47e000e930 tx=0x7f47e000edf0 comp rx=0 tx=0).stop 2024-04-25T23:49:03.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.943+0000 7f47f8535640 1 -- 172.21.15.57:0/3244519441 shutdown_connections 2024-04-25T23:49:03.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.943+0000 7f47f8535640 1 --2- 172.21.15.57:0/3244519441 >> v2:172.21.15.80:6816/2493457677 conn(0x7f47d40678b0 0x7f47d4069d70 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:03.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.943+0000 7f47f8535640 1 --2- 172.21.15.57:0/3244519441 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47f013f7c0 0x7f47f013b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:03.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.943+0000 7f47f8535640 1 --2- 172.21.15.57:0/3244519441 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f47f00b5310 0x7f47f013b380 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:03.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.943+0000 7f47f8535640 1 --2- 172.21.15.57:0/3244519441 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f47f00a9a70 0x7f47f0141f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:03.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.943+0000 7f47f8535640 1 -- 172.21.15.57:0/3244519441 >> 172.21.15.57:0/3244519441 conn(0x7f47f0067390 msgr2=0x7f47f013d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:49:03.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.943+0000 7f47f8535640 1 -- 172.21.15.57:0/3244519441 shutdown_connections 2024-04-25T23:49:03.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:03.943+0000 7f47f8535640 1 -- 172.21.15.57:0/3244519441 wait complete. 2024-04-25T23:49:03.953 DEBUG:teuthology.orchestra.run.smithi057:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph auth get-or-create client.libcephsqlite mon '"'"'profile simple-rados-client-with-blocklist'"'"' osd '"'"'allow rwx pool=cephsqlite'"'"' >> /etc/ceph/ceph.keyring' 2024-04-25T23:49:04.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.087+0000 7fb121e6a640 1 Processor -- start 2024-04-25T23:49:04.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.087+0000 7fb121e6a640 1 -- start start 2024-04-25T23:49:04.078 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.091+0000 7fb121e6a640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb11c0aea60 0x7fb11c072ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:04.078 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.091+0000 7fb121e6a640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fb11c0aee40 0x7fb11c143060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:04.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.091+0000 7fb121e6a640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fb11c1435a0 0x7fb11c06ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:04.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.091+0000 7fb121e6a640 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 -- 0x7fb11c0723b0 con 0x7fb11c1435a0 2024-04-25T23:49:04.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.091+0000 7fb121e6a640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fb11c072690 con 0x7fb11c0aea60 2024-04-25T23:49:04.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.091+0000 7fb121e6a640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7fb11c072ff0 con 0x7fb11c0aee40 2024-04-25T23:49:04.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.091+0000 7fb11affd640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fb11c0aee40 0x7fb11c143060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:04.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.091+0000 7fb11bfff640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fb11c1435a0 0x7fb11c06ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:04.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.091+0000 7fb11bfff640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fb11c1435a0 0x7fb11c06ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33740/0 (socket says 172.21.15.57:33740) 2024-04-25T23:49:04.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.091+0000 7fb11bfff640 1 -- 172.21.15.57:0/2291938513 learned_addr learned my addr 172.21.15.57:0/2291938513 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:49:04.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.091+0000 7fb11b7fe640 1 --2- 172.21.15.57:0/2291938513 >> v2:172.21.15.80:3300/0 conn(0x7fb11c0aea60 0x7fb11c072ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:04.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.091+0000 7fb11affd640 1 -- 172.21.15.57:0/2291938513 >> v2:172.21.15.80:3300/0 conn(0x7fb11c0aea60 msgr2=0x7fb11c072ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:49:04.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.091+0000 7fb11affd640 1 --2- 172.21.15.57:0/2291938513 >> v2:172.21.15.80:3300/0 conn(0x7fb11c0aea60 0x7fb11c072ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:04.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.091+0000 7fb11affd640 1 -- 172.21.15.57:0/2291938513 >> v2:172.21.15.57:3300/0 conn(0x7fb11c1435a0 msgr2=0x7fb11c06ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:49:04.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.091+0000 7fb11affd640 1 --2- 172.21.15.57:0/2291938513 >> v2:172.21.15.57:3300/0 conn(0x7fb11c1435a0 0x7fb11c06ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:04.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.091+0000 7fb11affd640 1 -- 172.21.15.57:0/2291938513 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb11c142900 con 0x7fb11c0aee40 2024-04-25T23:49:04.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.091+0000 7fb11affd640 1 --2- 172.21.15.57:0/2291938513 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fb11c0aee40 0x7fb11c143060 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb11c0af2c0 tx=0x7fb11001c3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=507699c6f2faa1a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:04.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.091+0000 7fb11a7fc640 1 -- 172.21.15.57:0/2291938513 <== mon.2 v2:172.21.15.57:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1100b0070 con 0x7fb11c0aee40 2024-04-25T23:49:04.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.091+0000 7fb11a7fc640 1 -- 172.21.15.57:0/2291938513 <== mon.2 v2:172.21.15.57:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1100ab600 con 0x7fb11c0aee40 2024-04-25T23:49:04.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.091+0000 7fb11a7fc640 1 -- 172.21.15.57:0/2291938513 <== mon.2 v2:172.21.15.57:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1100ab9d0 con 0x7fb11c0aee40 2024-04-25T23:49:04.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.095+0000 7fb121e6a640 1 -- 172.21.15.57:0/2291938513 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fb11c0aee40 msgr2=0x7fb11c143060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:04.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.095+0000 7fb121e6a640 1 --2- 172.21.15.57:0/2291938513 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fb11c0aee40 0x7fb11c143060 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb11c0af2c0 tx=0x7fb11001c3d0 comp rx=0 tx=0).stop 2024-04-25T23:49:04.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.095+0000 7fb121e6a640 1 -- 172.21.15.57:0/2291938513 shutdown_connections 2024-04-25T23:49:04.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.095+0000 7fb121e6a640 1 --2- 172.21.15.57:0/2291938513 >> v2:172.21.15.57:3300/0 conn(0x7fb11c1435a0 0x7fb11c06ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:04.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.095+0000 7fb121e6a640 1 --2- 172.21.15.57:0/2291938513 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fb11c0aee40 0x7fb11c143060 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:04.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.095+0000 7fb121e6a640 1 --2- 172.21.15.57:0/2291938513 >> v2:172.21.15.80:3300/0 conn(0x7fb11c0aea60 0x7fb11c072ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:04.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.095+0000 7fb121e6a640 1 -- 172.21.15.57:0/2291938513 >> 172.21.15.57:0/2291938513 conn(0x7fb11c0b4230 msgr2=0x7fb11c0b4640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:49:04.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.095+0000 7fb121e6a640 1 -- 172.21.15.57:0/2291938513 shutdown_connections 2024-04-25T23:49:04.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.095+0000 7fb121e6a640 1 -- 172.21.15.57:0/2291938513 wait complete. 2024-04-25T23:49:04.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.095+0000 7fb121e6a640 1 Processor -- start 2024-04-25T23:49:04.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.095+0000 7fb121e6a640 1 -- start start 2024-04-25T23:49:04.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.095+0000 7fb121e6a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb11c0aea60 0x7fb11c0ac970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:04.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.095+0000 7fb121e6a640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb11c1435a0 0x7fb11c137f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:04.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.095+0000 7fb121e6a640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fb11c1395b0 0x7fb11c13a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:04.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.095+0000 7fb121e6a640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fb11c13ffa0 con 0x7fb11c1435a0 2024-04-25T23:49:04.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.095+0000 7fb121e6a640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb11c13fe20 con 0x7fb11c0aea60 2024-04-25T23:49:04.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.095+0000 7fb121e6a640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7fb11c140120 con 0x7fb11c1395b0 2024-04-25T23:49:04.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.095+0000 7fb11affd640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb11c1435a0 0x7fb11c137f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:04.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.095+0000 7fb11affd640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb11c1435a0 0x7fb11c137f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33750/0 (socket says 172.21.15.57:33750) 2024-04-25T23:49:04.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.095+0000 7fb11affd640 1 -- 172.21.15.57:0/2791201287 learned_addr learned my addr 172.21.15.57:0/2791201287 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:49:04.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.095+0000 7fb11bfff640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fb11c1395b0 0x7fb11c13a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:04.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.095+0000 7fb11b7fe640 1 --2- 172.21.15.57:0/2791201287 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb11c0aea60 0x7fb11c0ac970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:04.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.095+0000 7fb11affd640 1 -- 172.21.15.57:0/2791201287 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fb11c1395b0 msgr2=0x7fb11c13a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:04.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.095+0000 7fb11affd640 1 --2- 172.21.15.57:0/2791201287 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fb11c1395b0 0x7fb11c13a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:04.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.095+0000 7fb11affd640 1 -- 172.21.15.57:0/2791201287 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb11c0aea60 msgr2=0x7fb11c0ac970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:04.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.095+0000 7fb11affd640 1 --2- 172.21.15.57:0/2791201287 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb11c0aea60 0x7fb11c0ac970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:04.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.095+0000 7fb11affd640 1 -- 172.21.15.57:0/2791201287 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb11c139a60 con 0x7fb11c1435a0 2024-04-25T23:49:04.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.095+0000 7fb11bfff640 1 --2- 172.21.15.57:0/2791201287 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fb11c1395b0 0x7fb11c13a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T23:49:04.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.095+0000 7fb11affd640 1 --2- 172.21.15.57:0/2791201287 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb11c1435a0 0x7fb11c137f00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb1100ae040 tx=0x7fb110019170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:04.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.095+0000 7fb103fff640 1 -- 172.21.15.57:0/2791201287 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1100b0040 con 0x7fb11c1435a0 2024-04-25T23:49:04.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.095+0000 7fb103fff640 1 -- 172.21.15.57:0/2791201287 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1100b9070 con 0x7fb11c1435a0 2024-04-25T23:49:04.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.095+0000 7fb103fff640 1 -- 172.21.15.57:0/2791201287 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1100aa9b0 con 0x7fb11c1435a0 2024-04-25T23:49:04.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.095+0000 7fb121e6a640 1 -- 172.21.15.57:0/2791201287 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb11c139cf0 con 0x7fb11c1435a0 2024-04-25T23:49:04.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.095+0000 7fb121e6a640 1 -- 172.21.15.57:0/2791201287 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb11c13ab90 con 0x7fb11c1435a0 2024-04-25T23:49:04.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.099+0000 7fb121e6a640 1 -- 172.21.15.57:0/2791201287 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb11c139ec0 con 0x7fb11c1435a0 2024-04-25T23:49:04.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.099+0000 7fb103fff640 1 -- 172.21.15.57:0/2791201287 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) ==== 87410+0+0 (secure 0 0 0) 0x7fb1100aab50 con 0x7fb11c1435a0 2024-04-25T23:49:04.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.099+0000 7fb103fff640 1 --2- 172.21.15.57:0/2791201287 >> v2:172.21.15.80:6816/2493457677 conn(0x7fb1080678b0 0x7fb108069d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:49:04.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.099+0000 7fb103fff640 1 -- 172.21.15.57:0/2791201287 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4439+0+0 (secure 0 0 0) 0x7fb1101203e0 con 0x7fb11c1435a0 2024-04-25T23:49:04.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.099+0000 7fb11b7fe640 1 --2- 172.21.15.57:0/2791201287 >> v2:172.21.15.80:6816/2493457677 conn(0x7fb1080678b0 0x7fb108069d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:49:04.088 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.099+0000 7fb11b7fe640 1 --2- 172.21.15.57:0/2791201287 >> v2:172.21.15.80:6816/2493457677 conn(0x7fb1080678b0 0x7fb108069d70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb10c00b990 tx=0x7fb10c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:49:04.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.103+0000 7fb103fff640 1 -- 172.21.15.57:0/2791201287 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb110016050 con 0x7fb11c1435a0 2024-04-25T23:49:04.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.323+0000 7fb121e6a640 1 -- 172.21.15.57:0/2791201287 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.libcephsqlite", "caps": ["mon", "profile simple-rados-client-with-blocklist", "osd", "allow rwx pool=cephsqlite"]} v 0) -- 0x7fb11c04f5b0 con 0x7fb11c1435a0 2024-04-25T23:49:04.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.331+0000 7fb103fff640 1 -- 172.21.15.57:0/2791201287 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.libcephsqlite", "caps": ["mon", "profile simple-rados-client-with-blocklist", "osd", "allow rwx pool=cephsqlite"]}]=0 v7) ==== 199+0+71 (secure 0 0 0) 0x7fb1100f1480 con 0x7fb11c1435a0 2024-04-25T23:49:04.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.331+0000 7fb121e6a640 1 -- 172.21.15.57:0/2791201287 >> v2:172.21.15.80:6816/2493457677 conn(0x7fb1080678b0 msgr2=0x7fb108069d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:04.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.331+0000 7fb121e6a640 1 --2- 172.21.15.57:0/2791201287 >> v2:172.21.15.80:6816/2493457677 conn(0x7fb1080678b0 0x7fb108069d70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb10c00b990 tx=0x7fb10c008040 comp rx=0 tx=0).stop 2024-04-25T23:49:04.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.331+0000 7fb121e6a640 1 -- 172.21.15.57:0/2791201287 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb11c1435a0 msgr2=0x7fb11c137f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:49:04.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.331+0000 7fb121e6a640 1 --2- 172.21.15.57:0/2791201287 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb11c1435a0 0x7fb11c137f00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb1100ae040 tx=0x7fb110019170 comp rx=0 tx=0).stop 2024-04-25T23:49:04.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.331+0000 7fb121e6a640 1 -- 172.21.15.57:0/2791201287 shutdown_connections 2024-04-25T23:49:04.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.331+0000 7fb121e6a640 1 --2- 172.21.15.57:0/2791201287 >> v2:172.21.15.80:6816/2493457677 conn(0x7fb1080678b0 0x7fb108069d70 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:04.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.331+0000 7fb121e6a640 1 --2- 172.21.15.57:0/2791201287 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fb11c1395b0 0x7fb11c13a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:04.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.331+0000 7fb121e6a640 1 --2- 172.21.15.57:0/2791201287 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb11c1435a0 0x7fb11c137f00 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:04.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.331+0000 7fb121e6a640 1 --2- 172.21.15.57:0/2791201287 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb11c0aea60 0x7fb11c0ac970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:49:04.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.331+0000 7fb121e6a640 1 -- 172.21.15.57:0/2791201287 >> 172.21.15.57:0/2791201287 conn(0x7fb11c0b4230 msgr2=0x7fb11c06ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:49:04.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.331+0000 7fb121e6a640 1 -- 172.21.15.57:0/2791201287 shutdown_connections 2024-04-25T23:49:04.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:49:04.331+0000 7fb121e6a640 1 -- 172.21.15.57:0/2791201287 wait complete. 2024-04-25T23:49:04.336 INFO:teuthology.run_tasks:Running task exec... 2024-04-25T23:49:04.344 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T23:49:04.345 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi057.front.sepia.ceph.com 2024-04-25T23:49:04.345 DEBUG:teuthology.orchestra.run.smithi057:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph_test_libcephsqlite --id libcephsqlite --no-log-to-stderr' 2024-04-25T23:49:04.383 INFO:teuthology.orchestra.run.smithi057.stdout:[==========] Running 20 tests from 1 test suite. 2024-04-25T23:49:04.383 INFO:teuthology.orchestra.run.smithi057.stdout:[----------] Global test environment set-up. 2024-04-25T23:49:04.383 INFO:teuthology.orchestra.run.smithi057.stdout:[----------] 20 tests from CephSQLiteTest 2024-04-25T23:49:09.390 INFO:teuthology.orchestra.run.smithi057.stdout:[ RUN ] CephSQLiteTest.Create 2024-04-25T23:49:09.406 INFO:teuthology.orchestra.run.smithi057.stdout:using database: file:cephsqlite:/2429344d-e95a-473d-bc8d-46aaa0f8b2a9.db?vfs=ceph 2024-04-25T23:49:09.407 INFO:teuthology.orchestra.run.smithi057.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-04-25T23:49:09.416 INFO:teuthology.orchestra.run.smithi057.stdout:CREATE TABLE foo (a INT); 2024-04-25T23:49:09.447 INFO:teuthology.orchestra.run.smithi057.stdout:[ OK ] CephSQLiteTest.Create (58 ms) 2024-04-25T23:49:09.448 INFO:teuthology.orchestra.run.smithi057.stdout:[ RUN ] CephSQLiteTest.InsertBulk4096 2024-04-25T23:49:09.461 INFO:teuthology.orchestra.run.smithi057.stdout:using database: file:cephsqlite:/36e0ed20-cd36-460b-aa12-e6bc2711a2d0.db?vfs=ceph 2024-04-25T23:49:09.461 INFO:teuthology.orchestra.run.smithi057.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-04-25T23:49:09.469 INFO:teuthology.orchestra.run.smithi057.stdout:PRAGMA page_size = 4096;CREATE TABLE foo (a INT);WITH RECURSIVE c(x) AS ( VALUES(1) UNION ALL SELECT x+1 FROM c )INSERT INTO foo (a) SELECT RANDOM() FROM c LIMIT 1000000;PRAGMA page_size; 2024-04-25T23:49:13.131 INFO:teuthology.orchestra.run.smithi057.stdout:[ OK ] CephSQLiteTest.InsertBulk4096 (3683 ms) 2024-04-25T23:49:13.131 INFO:teuthology.orchestra.run.smithi057.stdout:[ RUN ] CephSQLiteTest.InsertBulk 2024-04-25T23:49:13.144 INFO:teuthology.orchestra.run.smithi057.stdout:using database: file:cephsqlite:/740e02b6-6c92-4bde-be04-8f45ee078323.db?vfs=ceph 2024-04-25T23:49:13.144 INFO:teuthology.orchestra.run.smithi057.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-04-25T23:49:13.150 INFO:teuthology.orchestra.run.smithi057.stdout:CREATE TABLE foo (a INT);WITH RECURSIVE c(x) AS ( VALUES(1) UNION ALL SELECT x+1 FROM c )INSERT INTO foo (a) SELECT RANDOM() FROM c LIMIT 1000000; 2024-04-25T23:49:13.758 INFO:teuthology.orchestra.run.smithi057.stdout:[ OK ] CephSQLiteTest.InsertBulk (627 ms) 2024-04-25T23:49:13.758 INFO:teuthology.orchestra.run.smithi057.stdout:[ RUN ] CephSQLiteTest.UpdateBulk 2024-04-25T23:49:13.767 INFO:teuthology.orchestra.run.smithi057.stdout:using database: file:cephsqlite:/39b6c434-5677-4a3f-9b6c-1941188313eb.db?vfs=ceph 2024-04-25T23:49:13.767 INFO:teuthology.orchestra.run.smithi057.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-04-25T23:49:13.771 INFO:teuthology.orchestra.run.smithi057.stdout:CREATE TABLE foo (a INT);WITH RECURSIVE c(x) AS ( VALUES(1) UNION ALL SELECT x+1 FROM c )INSERT INTO foo (a) SELECT x FROM c LIMIT 1000000;SELECT SUM(a) FROM foo;UPDATE foo SET a = a+a;SELECT SUM(a) FROM foo; 2024-04-25T23:49:14.755 INFO:teuthology.orchestra.run.smithi057.stdout:[ OK ] CephSQLiteTest.UpdateBulk (997 ms) 2024-04-25T23:49:14.755 INFO:teuthology.orchestra.run.smithi057.stdout:[ RUN ] CephSQLiteTest.InsertRate 2024-04-25T23:49:14.767 INFO:teuthology.orchestra.run.smithi057.stdout:using database: file:cephsqlite:/ca3666dc-a78b-44a8-a527-9ec81b51d2db.db?vfs=ceph 2024-04-25T23:49:14.767 INFO:teuthology.orchestra.run.smithi057.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-04-25T23:49:14.776 INFO:teuthology.orchestra.run.smithi057.stdout:CREATE TABLE foo (a INT);INSERT INTO foo (a) VALUES (RANDOM()); 2024-04-25T23:49:18.494 INFO:teuthology.orchestra.run.smithi057.stdout:transactions per second: 27.0858 2024-04-25T23:49:18.495 INFO:teuthology.orchestra.run.smithi057.stdout:[ OK ] CephSQLiteTest.InsertRate (3741 ms) 2024-04-25T23:49:18.495 INFO:teuthology.orchestra.run.smithi057.stdout:[ RUN ] CephSQLiteTest.DatabaseShrink 2024-04-25T23:49:18.510 INFO:teuthology.orchestra.run.smithi057.stdout:using database: file:cephsqlite:/98028126-12a8-430c-9164-c9e3855b1fe9.db?vfs=ceph 2024-04-25T23:49:18.511 INFO:teuthology.orchestra.run.smithi057.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-04-25T23:49:18.518 INFO:teuthology.orchestra.run.smithi057.stdout:CREATE TABLE foo (a INT);WITH RECURSIVE c(x) AS ( VALUES(1) UNION ALL SELECT x+1 FROM c )INSERT INTO foo (a) SELECT x FROM c LIMIT 1000000;DELETE FROM foo WHERE RANDOM()%4 < 3;VACUUM; 2024-04-25T23:49:20.154 INFO:teuthology.orchestra.run.smithi057.stdout:[ OK ] CephSQLiteTest.DatabaseShrink (1658 ms) 2024-04-25T23:49:20.154 INFO:teuthology.orchestra.run.smithi057.stdout:[ RUN ] CephSQLiteTest.InsertExclusiveRate 2024-04-25T23:49:20.170 INFO:teuthology.orchestra.run.smithi057.stdout:using database: file:cephsqlite:/b23ee357-e09b-40a4-ac91-141fdcaa1426.db?vfs=ceph 2024-04-25T23:49:20.170 INFO:teuthology.orchestra.run.smithi057.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-04-25T23:49:20.177 INFO:teuthology.orchestra.run.smithi057.stdout:PRAGMA locking_mode=EXCLUSIVE;CREATE TABLE foo (a INT);INSERT INTO foo (a) VALUES (RANDOM()); 2024-04-25T23:49:21.490 INFO:teuthology.orchestra.run.smithi057.stdout:transactions per second: 77.3979 2024-04-25T23:49:21.496 INFO:teuthology.orchestra.run.smithi057.stdout:[ OK ] CephSQLiteTest.InsertExclusiveRate (1342 ms) 2024-04-25T23:49:21.496 INFO:teuthology.orchestra.run.smithi057.stdout:[ RUN ] CephSQLiteTest.InsertExclusiveWALRate 2024-04-25T23:49:21.515 INFO:teuthology.orchestra.run.smithi057.stdout:using database: file:cephsqlite:/2aa40989-fddc-4f47-b268-be7d7fd321a1.db?vfs=ceph 2024-04-25T23:49:21.515 INFO:teuthology.orchestra.run.smithi057.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-04-25T23:49:21.523 INFO:teuthology.orchestra.run.smithi057.stdout:PRAGMA locking_mode=EXCLUSIVE;PRAGMA journal_mode=WAL;CREATE TABLE foo (a INT);INSERT INTO foo (a) VALUES (RANDOM()); 2024-04-25T23:49:22.182 INFO:teuthology.orchestra.run.smithi057.stdout:transactions per second: 168.916 2024-04-25T23:49:22.220 INFO:teuthology.orchestra.run.smithi057.stdout:[ OK ] CephSQLiteTest.InsertExclusiveWALRate (725 ms) 2024-04-25T23:49:22.220 INFO:teuthology.orchestra.run.smithi057.stdout:[ RUN ] CephSQLiteTest.WALTransactionSync 2024-04-25T23:49:22.232 INFO:teuthology.orchestra.run.smithi057.stdout:using database: file:cephsqlite:/b7808858-2681-4dcb-a07d-fa20d81a8fce.db?vfs=ceph 2024-04-25T23:49:22.232 INFO:teuthology.orchestra.run.smithi057.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-04-25T23:49:22.239 INFO:teuthology.orchestra.run.smithi057.stdout:PRAGMA locking_mode=EXCLUSIVE;PRAGMA journal_mode=WAL;CREATE TABLE foo (a INT);INSERT INTO perf (v) VALUES (ceph_perf());BEGIN TRANSACTION;INSERT INTO foo (a) VALUES (RANDOM());END TRANSACTION;INSERT INTO perf (v) VALUES (ceph_perf());SELECT a.atom-b.atom FROM p AS a, p AS b WHERE a.i = ? AND b.i = ? AND a.fullkey = '$.libcephsqlite_vfs.opf_sync.avgcount' AND b.fullkey = '$.libcephsqlite_vfs.opf_sync.avgcount'; 2024-04-25T23:49:22.328 INFO:teuthology.orchestra.run.smithi057.stdout:[ OK ] CephSQLiteTest.WALTransactionSync (108 ms) 2024-04-25T23:49:22.328 INFO:teuthology.orchestra.run.smithi057.stdout:[ RUN ] CephSQLiteTest.PersistTransactionSync 2024-04-25T23:49:22.338 INFO:teuthology.orchestra.run.smithi057.stdout:using database: file:cephsqlite:/4f11ae7b-63c7-42c8-9dbb-77c8c371b010.db?vfs=ceph 2024-04-25T23:49:22.338 INFO:teuthology.orchestra.run.smithi057.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-04-25T23:49:22.343 INFO:teuthology.orchestra.run.smithi057.stdout:BEGIN TRANSACTION;CREATE TABLE foo (a INT);INSERT INTO foo (a) VALUES (RANDOM());END TRANSACTION;INSERT INTO perf (v) VALUES (ceph_perf());SELECT a.atom-b.atom FROM p AS a, p AS b WHERE a.i = ? AND b.i = ? AND a.fullkey = '$.libcephsqlite_vfs.opf_sync.avgcount' AND b.fullkey = '$.libcephsqlite_vfs.opf_sync.avgcount'; 2024-04-25T23:49:22.373 INFO:teuthology.orchestra.run.smithi057.stdout:[ OK ] CephSQLiteTest.PersistTransactionSync (44 ms) 2024-04-25T23:49:22.373 INFO:teuthology.orchestra.run.smithi057.stdout:[ RUN ] CephSQLiteTest.InsertExclusiveLock 2024-04-25T23:49:22.384 INFO:teuthology.orchestra.run.smithi057.stdout:using database: file:cephsqlite:/d79f50f9-e723-4f99-910f-88508d20a60e.db?vfs=ceph 2024-04-25T23:49:22.384 INFO:teuthology.orchestra.run.smithi057.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-04-25T23:49:22.390 INFO:teuthology.orchestra.run.smithi057.stdout:PRAGMA locking_mode=EXCLUSIVE;CREATE TABLE foo (a INT);INSERT INTO foo (a) VALUES (RANDOM());INSERT INTO perf (v) VALUES (ceph_perf());SELECT a.atom, b.atom, a.atom-b.atom FROM p AS a, p AS b WHERE a.i = ? AND b.i = ? AND a.fullkey = '$.libcephsqlite_vfs.opf_lock.avgcount' AND b.fullkey = '$.libcephsqlite_vfs.opf_lock.avgcount';SELECT a.atom, b.atom, a.atom-b.atom FROM p AS a, p AS b WHERE a.i = ? AND b.i = ? AND a.fullkey = '$.libcephsqlite_striper.lock' AND b.fullkey = '$.libcephsqlite_striper.lock'; 2024-04-25T23:49:22.422 INFO:teuthology.orchestra.run.smithi057.stdout:[ OK ] CephSQLiteTest.InsertExclusiveLock (49 ms) 2024-04-25T23:49:22.422 INFO:teuthology.orchestra.run.smithi057.stdout:[ RUN ] CephSQLiteTest.TransactionSizeUpdate 2024-04-25T23:49:22.433 INFO:teuthology.orchestra.run.smithi057.stdout:using database: file:cephsqlite:/6a2a7e3d-a9a3-4e7b-9626-abc1c7515132.db?vfs=ceph 2024-04-25T23:49:22.434 INFO:teuthology.orchestra.run.smithi057.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-04-25T23:49:22.440 INFO:teuthology.orchestra.run.smithi057.stdout:BEGIN TRANSACTION;CREATE TABLE foo (a INT);INSERT INTO foo (a) VALUES (RANDOM());END TRANSACTION;INSERT INTO perf (v) VALUES (ceph_perf());SELECT a.atom, b.atom, a.atom-b.atom FROM p AS a, p AS b WHERE a.i = ? AND b.i = ? AND a.fullkey = '$.libcephsqlite_striper.update_size' AND b.fullkey = '$.libcephsqlite_striper.update_size'; 2024-04-25T23:49:22.485 INFO:teuthology.orchestra.run.smithi057.stdout:[ OK ] CephSQLiteTest.TransactionSizeUpdate (64 ms) 2024-04-25T23:49:22.485 INFO:teuthology.orchestra.run.smithi057.stdout:[ RUN ] CephSQLiteTest.AllocatedGrowth 2024-04-25T23:49:22.703 INFO:teuthology.orchestra.run.smithi057.stdout:using database: file:cephsqlite:/749cea53-85b5-4cce-9260-9ec742601035.db?vfs=ceph 2024-04-25T23:49:22.704 INFO:teuthology.orchestra.run.smithi057.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-04-25T23:49:22.710 INFO:teuthology.orchestra.run.smithi057.stdout:CREATE TABLE foo (a BLOB);WITH RECURSIVE c(x) AS ( VALUES(1) UNION ALL SELECT x+1 FROM c )INSERT INTO foo (a) SELECT RANDOMBLOB(1<<20) FROM c LIMIT 1024;INSERT INTO perf (v) VALUES (ceph_perf());SELECT a.atom, b.atom, a.atom-b.atom FROM p AS a, p AS b WHERE a.i = ? AND b.i = ? AND a.fullkey = '$.libcephsqlite_striper.update_allocated' AND b.fullkey = '$.libcephsqlite_striper.update_allocated'; 2024-04-25T23:49:31.066 INFO:teuthology.orchestra.run.smithi057.stdout:[ OK ] CephSQLiteTest.AllocatedGrowth (8581 ms) 2024-04-25T23:49:31.066 INFO:teuthology.orchestra.run.smithi057.stdout:[ RUN ] CephSQLiteTest.DeleteBulk 2024-04-25T23:49:31.080 INFO:teuthology.orchestra.run.smithi057.stdout:using database: file:cephsqlite:/3400a5e9-7036-4b9d-b6db-6fbc2746b755.db?vfs=ceph 2024-04-25T23:49:31.080 INFO:teuthology.orchestra.run.smithi057.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-04-25T23:49:31.088 INFO:teuthology.orchestra.run.smithi057.stdout:CREATE TABLE foo (a INT);WITH RECURSIVE c(x) AS ( VALUES(1) UNION ALL SELECT x+1 FROM c )INSERT INTO foo (a) SELECT x FROM c LIMIT 1000000;DELETE FROM foo WHERE RANDOM()%2 == 0; 2024-04-25T23:49:32.265 INFO:teuthology.orchestra.run.smithi057.stdout:[ OK ] CephSQLiteTest.DeleteBulk (1199 ms) 2024-04-25T23:49:32.265 INFO:teuthology.orchestra.run.smithi057.stdout:[ RUN ] CephSQLiteTest.DropMassive 2024-04-25T23:49:32.278 INFO:teuthology.orchestra.run.smithi057.stdout:using database: file:cephsqlite:/66d6a55e-b343-4276-a07f-12e9a6a34650.db?vfs=ceph 2024-04-25T23:49:32.279 INFO:teuthology.orchestra.run.smithi057.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-04-25T23:49:32.284 INFO:teuthology.orchestra.run.smithi057.stdout:CREATE TABLE foo (a BLOB);WITH RECURSIVE c(x) AS ( VALUES(1) UNION ALL SELECT x+1 FROM c )INSERT INTO foo (a) SELECT RANDOMBLOB(1<<20) FROM c LIMIT 1024;DROP TABLE foo;VACUUM;INSERT INTO perf (v) VALUES (ceph_perf());SELECT a.atom, b.atom FROM p AS a, p AS b WHERE a.i = ? AND b.i = ? AND a.fullkey = '$.libcephsqlite_striper.shrink' AND b.fullkey = '$.libcephsqlite_striper.shrink';SELECT a.atom-b.atom FROM p AS a, p AS b WHERE a.i = ? AND b.i = ? AND a.fullkey = '$.libcephsqlite_striper.shrink_bytes' AND b.fullkey = '$.libcephsqlite_striper.shrink_bytes'; 2024-04-25T23:50:11.020 INFO:teuthology.orchestra.run.smithi057.stdout:[ OK ] CephSQLiteTest.DropMassive (38755 ms) 2024-04-25T23:50:11.021 INFO:teuthology.orchestra.run.smithi057.stdout:[ RUN ] CephSQLiteTest.InsertMassiveVerify 2024-04-25T23:50:11.033 INFO:teuthology.orchestra.run.smithi057.stdout:using database: file:cephsqlite:/460f24c7-7a33-4f80-972e-f88088851c77.db?vfs=ceph 2024-04-25T23:50:11.034 INFO:teuthology.orchestra.run.smithi057.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-04-25T23:50:11.038 INFO:teuthology.orchestra.run.smithi057.stdout:CREATE TABLE foo (a BLOB);CREATE TEMPORARY TABLE bar (a BLOB);WITH RECURSIVE c(x) AS ( VALUES(1) UNION ALL SELECT x+1 FROM c )INSERT INTO bar (a) SELECT RANDOMBLOB(1<<20) FROM c LIMIT 1024;SELECT a FROM bar;INSERT INTO foo (a) SELECT a FROM bar;SELECT a FROM foo; 2024-04-25T23:50:26.729 INFO:teuthology.orchestra.run.smithi057.stdout:[ OK ] CephSQLiteTest.InsertMassiveVerify (15709 ms) 2024-04-25T23:50:26.729 INFO:teuthology.orchestra.run.smithi057.stdout:[ RUN ] CephSQLiteTest.PerfValid 2024-04-25T23:50:26.744 INFO:teuthology.orchestra.run.smithi057.stdout:using database: file:cephsqlite:/083fd62c-d2b4-44c8-a038-164e42795ec3.db?vfs=ceph 2024-04-25T23:50:26.744 INFO:teuthology.orchestra.run.smithi057.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-04-25T23:50:26.755 INFO:teuthology.orchestra.run.smithi057.stdout:SELECT json_valid(ceph_perf()); 2024-04-25T23:50:26.756 INFO:teuthology.orchestra.run.smithi057.stdout:[ OK ] CephSQLiteTest.PerfValid (28 ms) 2024-04-25T23:50:26.756 INFO:teuthology.orchestra.run.smithi057.stdout:[ RUN ] CephSQLiteTest.StatusValid 2024-04-25T23:50:26.768 INFO:teuthology.orchestra.run.smithi057.stdout:using database: file:cephsqlite:/ffa4d033-85a3-4f23-8a85-c246eeae8e1d.db?vfs=ceph 2024-04-25T23:50:26.768 INFO:teuthology.orchestra.run.smithi057.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-04-25T23:50:26.776 INFO:teuthology.orchestra.run.smithi057.stdout:SELECT json_valid(ceph_status()); 2024-04-25T23:50:26.776 INFO:teuthology.orchestra.run.smithi057.stdout:[ OK ] CephSQLiteTest.StatusValid (20 ms) 2024-04-25T23:50:26.776 INFO:teuthology.orchestra.run.smithi057.stdout:[ RUN ] CephSQLiteTest.CurrentTime 2024-04-25T23:50:26.790 INFO:teuthology.orchestra.run.smithi057.stdout:using database: file:cephsqlite:/8e49500d-503d-4823-9ee3-56963e3f5aae.db?vfs=ceph 2024-04-25T23:50:26.790 INFO:teuthology.orchestra.run.smithi057.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-04-25T23:50:26.806 INFO:teuthology.orchestra.run.smithi057.stdout:SELECT strftime('%s', 'now'); 2024-04-25T23:50:26.807 INFO:teuthology.orchestra.run.smithi057.stdout:[ OK ] CephSQLiteTest.CurrentTime (31 ms) 2024-04-25T23:50:26.807 INFO:teuthology.orchestra.run.smithi057.stdout:[ RUN ] CephSQLiteTest.StatusFields 2024-04-25T23:50:26.817 INFO:teuthology.orchestra.run.smithi057.stdout:using database: file:cephsqlite:/627ce91d-94de-49ae-8114-4dfce0e636a0.db?vfs=ceph 2024-04-25T23:50:26.817 INFO:teuthology.orchestra.run.smithi057.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-04-25T23:50:26.823 INFO:teuthology.orchestra.run.smithi057.stdout:SELECT json_extract(ceph_status(), '$.addr');SELECT json_extract(ceph_status(), '$.id'); 2024-04-25T23:50:26.823 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/3962177883 2024-04-25T23:50:26.823 INFO:teuthology.orchestra.run.smithi057.stdout:4419 2024-04-25T23:50:26.823 INFO:teuthology.orchestra.run.smithi057.stdout:[ OK ] CephSQLiteTest.StatusFields (16 ms) 2024-04-25T23:50:26.823 INFO:teuthology.orchestra.run.smithi057.stdout:[----------] 20 tests from CephSQLiteTest (77435 ms total) 2024-04-25T23:50:26.823 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-25T23:50:26.823 INFO:teuthology.orchestra.run.smithi057.stdout:[----------] Global test environment tear-down 2024-04-25T23:50:26.823 INFO:teuthology.orchestra.run.smithi057.stdout:[==========] 20 tests from 1 test suite ran. (82442 ms total) 2024-04-25T23:50:26.824 INFO:teuthology.orchestra.run.smithi057.stdout:[ PASSED ] 20 tests. 2024-04-25T23:50:26.864 INFO:teuthology.run_tasks:Running task workunit... 2024-04-25T23:50:26.886 INFO:tasks.workunit:Pulling workunits from ref b22e2ebdeb24376882b7bda2a7329c8cccc2276a 2024-04-25T23:50:26.887 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-25T23:50:26.887 DEBUG:teuthology.orchestra.run.smithi057:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-25T23:50:26.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T23:50:26.913 INFO:teuthology.orchestra.run.smithi057.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-25T23:50:26.914 DEBUG:teuthology.orchestra.run.smithi057:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-04-25T23:50:26.960 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-04-25T23:50:26.960 DEBUG:teuthology.orchestra.run.smithi057:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-04-25T23:50:27.008 INFO:tasks.workunit:timeout=3h 2024-04-25T23:50:27.008 INFO:tasks.workunit:cleanup=True 2024-04-25T23:50:27.008 DEBUG:teuthology.orchestra.run.smithi057:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout b22e2ebdeb24376882b7bda2a7329c8cccc2276a 2024-04-25T23:50:27.058 INFO:tasks.workunit.client.0.smithi057.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-25T23:51:57.244 INFO:tasks.workunit.client.0.smithi057.stderr:Updating files: 96% (11995/12492) Updating files: 97% (12118/12492) Updating files: 98% (12243/12492) Updating files: 99% (12368/12492) Updating files: 100% (12492/12492) Updating files: 100% (12492/12492), done. 2024-04-25T23:51:57.700 INFO:tasks.workunit.client.0.smithi057.stderr:Note: switching to 'b22e2ebdeb24376882b7bda2a7329c8cccc2276a'. 2024-04-25T23:51:57.700 INFO:tasks.workunit.client.0.smithi057.stderr: 2024-04-25T23:51:57.700 INFO:tasks.workunit.client.0.smithi057.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-25T23:51:57.700 INFO:tasks.workunit.client.0.smithi057.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-25T23:51:57.700 INFO:tasks.workunit.client.0.smithi057.stderr:state without impacting any branches by switching back to a branch. 2024-04-25T23:51:57.700 INFO:tasks.workunit.client.0.smithi057.stderr: 2024-04-25T23:51:57.700 INFO:tasks.workunit.client.0.smithi057.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-25T23:51:57.700 INFO:tasks.workunit.client.0.smithi057.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-25T23:51:57.700 INFO:tasks.workunit.client.0.smithi057.stderr: 2024-04-25T23:51:57.700 INFO:tasks.workunit.client.0.smithi057.stderr: git switch -c 2024-04-25T23:51:57.701 INFO:tasks.workunit.client.0.smithi057.stderr: 2024-04-25T23:51:57.701 INFO:tasks.workunit.client.0.smithi057.stderr:Or undo this operation with: 2024-04-25T23:51:57.701 INFO:tasks.workunit.client.0.smithi057.stderr: 2024-04-25T23:51:57.701 INFO:tasks.workunit.client.0.smithi057.stderr: git switch - 2024-04-25T23:51:57.701 INFO:tasks.workunit.client.0.smithi057.stderr: 2024-04-25T23:51:57.701 INFO:tasks.workunit.client.0.smithi057.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-25T23:51:57.701 INFO:tasks.workunit.client.0.smithi057.stderr: 2024-04-25T23:51:57.701 INFO:tasks.workunit.client.0.smithi057.stderr:HEAD is now at b22e2ebdeb2 Merge pull request #57078 from adk3798/squid-cephadm-no-downgrade 2024-04-25T23:51:57.705 DEBUG:teuthology.orchestra.run.smithi057:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-25T23:51:57.754 INFO:tasks.workunit.client.0.smithi057.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-25T23:51:57.758 INFO:tasks.workunit.client.0.smithi057.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-25T23:51:57.758 INFO:tasks.workunit.client.0.smithi057.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-25T23:51:58.611 INFO:tasks.workunit.client.0.smithi057.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-25T23:51:58.714 INFO:tasks.workunit.client.0.smithi057.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-25T23:51:58.771 INFO:tasks.workunit.client.0.smithi057.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-25T23:51:58.773 INFO:tasks.workunit.client.0.smithi057.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-25T23:51:58.773 INFO:tasks.workunit.client.0.smithi057.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-25T23:51:58.827 INFO:tasks.workunit.client.0.smithi057.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-25T23:51:58.832 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-04-25T23:51:58.832 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-25T23:51:58.881 INFO:tasks.workunit:Running workunits matching rados/test_libcephsqlite.sh on client.0... 2024-04-25T23:51:58.882 INFO:tasks.workunit:Running workunit rados/test_libcephsqlite.sh... 2024-04-25T23:51:58.882 DEBUG:teuthology.orchestra.run.smithi057:workunit test rados/test_libcephsqlite.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b22e2ebdeb24376882b7bda2a7329c8cccc2276a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 CEPH_ARGS='--id libcephsqlite --no-log-to-stderr' adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_libcephsqlite.sh cephsqlite 2024-04-25T23:51:58.933 INFO:tasks.workunit.client.0.smithi057.stderr:+ pool=cephsqlite 2024-04-25T23:51:58.934 INFO:tasks.workunit.client.0.smithi057.stderr:++ basename /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_libcephsqlite.sh 2024-04-25T23:51:58.934 INFO:tasks.workunit.client.0.smithi057.stderr:+ ns=test_libcephsqlite.sh 2024-04-25T23:51:58.935 INFO:tasks.workunit.client.0.smithi057.stderr:+ striper rm baz.db 2024-04-25T23:51:58.935 INFO:tasks.workunit.client.0.smithi057.stderr:+ rados --pool=cephsqlite --namespace=test_libcephsqlite.sh --striper rm baz.db 2024-04-25T23:51:58.997 INFO:tasks.workunit.client.0.smithi057.stderr:+ sqlite 2024-04-25T23:51:58.997 INFO:tasks.workunit.client.0.smithi057.stderr:+ background= 2024-04-25T23:51:58.997 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' '' = b ']' 2024-04-25T23:51:58.998 INFO:tasks.workunit.client.0.smithi057.stderr:++ cat 2024-04-25T23:51:58.999 INFO:tasks.workunit.client.0.smithi057.stderr:+ a='create table if not exists foo (a INT); 2024-04-25T23:51:58.999 INFO:tasks.workunit.client.0.smithi057.stderr:insert into foo (a) values (RANDOM()); 2024-04-25T23:51:58.999 INFO:tasks.workunit.client.0.smithi057.stderr:drop table foo;' 2024-04-25T23:51:58.999 INFO:tasks.workunit.client.0.smithi057.stderr:+ printf %s 'create table if not exists foo (a INT); 2024-04-25T23:51:58.999 INFO:tasks.workunit.client.0.smithi057.stderr:insert into foo (a) values (RANDOM()); 2024-04-25T23:51:58.999 INFO:tasks.workunit.client.0.smithi057.stderr:drop table foo;' 2024-04-25T23:51:58.999 INFO:tasks.workunit.client.0.smithi057.stderr:create table if not exists foo (a INT); 2024-04-25T23:51:58.999 INFO:tasks.workunit.client.0.smithi057.stderr:insert into foo (a) values (RANDOM()); 2024-04-25T23:51:58.999 INFO:tasks.workunit.client.0.smithi057.stderr:drop table foo;+ '[' '' '!=' b ']' 2024-04-25T23:51:58.999 INFO:tasks.workunit.client.0.smithi057.stderr:+ wait 2024-04-25T23:51:58.999 INFO:tasks.workunit.client.0.smithi057.stderr:+ sqlite3 -cmd '.output /dev/null' -cmd '.load libcephsqlite.so' -cmd 'pragma journal_mode = PERSIST' -cmd '.open file:///cephsqlite:test_libcephsqlite.sh/baz.db?vfs=ceph' -cmd '.output stdout' 2024-04-25T23:51:59.370 INFO:tasks.workunit.client.0.smithi057.stderr: 2024-04-25T23:51:59.370 INFO:tasks.workunit.client.0.smithi057.stderr:real 0m0.373s 2024-04-25T23:51:59.370 INFO:tasks.workunit.client.0.smithi057.stderr:user 0m0.054s 2024-04-25T23:51:59.370 INFO:tasks.workunit.client.0.smithi057.stderr:sys 0m0.045s 2024-04-25T23:51:59.370 INFO:tasks.workunit.client.0.smithi057.stderr:+ striper stat baz.db 2024-04-25T23:51:59.370 INFO:tasks.workunit.client.0.smithi057.stderr:+ rados --pool=cephsqlite --namespace=test_libcephsqlite.sh --striper stat baz.db 2024-04-25T23:51:59.417 INFO:tasks.workunit.client.0.smithi057.stdout:cephsqlite/baz.db mtime 2024-04-25T23:51:59.000000+0000, size 8192 2024-04-25T23:51:59.420 INFO:tasks.workunit.client.0.smithi057.stderr:+ striper rm baz.db 2024-04-25T23:51:59.420 INFO:tasks.workunit.client.0.smithi057.stderr:+ rados --pool=cephsqlite --namespace=test_libcephsqlite.sh --striper rm baz.db 2024-04-25T23:51:59.472 INFO:tasks.workunit.client.0.smithi057.stderr:+ sqlite 2024-04-25T23:51:59.472 INFO:tasks.workunit.client.0.smithi057.stderr:++ repeat 10 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:51:59.472 INFO:tasks.workunit.client.0.smithi057.stderr:++ n=10 2024-04-25T23:51:59.472 INFO:tasks.workunit.client.0.smithi057.stderr:++ shift 2024-04-25T23:51:59.472 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i = 0 )) 2024-04-25T23:51:59.473 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 10 )) 2024-04-25T23:51:59.473 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:51:59.473 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:51:59.473 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 10 )) 2024-04-25T23:51:59.473 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:51:59.473 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:51:59.473 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 10 )) 2024-04-25T23:51:59.473 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:51:59.473 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:51:59.473 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 10 )) 2024-04-25T23:51:59.473 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:51:59.473 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:51:59.473 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 10 )) 2024-04-25T23:51:59.473 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:51:59.473 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:51:59.474 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 10 )) 2024-04-25T23:51:59.474 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:51:59.474 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:51:59.474 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 10 )) 2024-04-25T23:51:59.474 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:51:59.474 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:51:59.474 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 10 )) 2024-04-25T23:51:59.474 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:51:59.474 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:51:59.474 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 10 )) 2024-04-25T23:51:59.474 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:51:59.474 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:51:59.474 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 10 )) 2024-04-25T23:51:59.474 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:51:59.475 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:51:59.475 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 10 )) 2024-04-25T23:51:59.475 INFO:tasks.workunit.client.0.smithi057.stderr:+ background= 2024-04-25T23:51:59.475 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' '' = b ']' 2024-04-25T23:51:59.475 INFO:tasks.workunit.client.0.smithi057.stderr:++ cat 2024-04-25T23:51:59.475 INFO:tasks.workunit.client.0.smithi057.stderr:+ a='CREATE TABLE IF NOT EXISTS rand(text BLOB NOT NULL); 2024-04-25T23:51:59.475 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:51:59.475 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:51:59.475 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:51:59.475 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:51:59.475 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:51:59.475 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:51:59.475 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:51:59.475 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:51:59.476 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:51:59.476 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:51:59.476 INFO:tasks.workunit.client.0.smithi057.stderr:SELECT LENGTH(text) FROM rand; 2024-04-25T23:51:59.476 INFO:tasks.workunit.client.0.smithi057.stderr:DROP TABLE rand;' 2024-04-25T23:51:59.476 INFO:tasks.workunit.client.0.smithi057.stderr:+ printf %s 'CREATE TABLE IF NOT EXISTS rand(text BLOB NOT NULL); 2024-04-25T23:51:59.476 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:51:59.476 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:51:59.476 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:51:59.476 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:51:59.476 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:51:59.476 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:51:59.476 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:51:59.476 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:51:59.476 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:51:59.476 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:51:59.477 INFO:tasks.workunit.client.0.smithi057.stderr:SELECT LENGTH(text) FROM rand; 2024-04-25T23:51:59.477 INFO:tasks.workunit.client.0.smithi057.stderr:DROP TABLE rand;' 2024-04-25T23:51:59.477 INFO:tasks.workunit.client.0.smithi057.stderr:CREATE TABLE IF NOT EXISTS rand(text BLOB NOT NULL); 2024-04-25T23:51:59.477 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:51:59.477 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:51:59.477 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:51:59.477 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:51:59.477 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:51:59.477 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:51:59.477 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:51:59.477 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:51:59.477 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:51:59.477 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:51:59.477 INFO:tasks.workunit.client.0.smithi057.stderr:SELECT LENGTH(text) FROM rand; 2024-04-25T23:51:59.477 INFO:tasks.workunit.client.0.smithi057.stderr:DROP TABLE rand;+ '[' '' '!=' b ']' 2024-04-25T23:51:59.478 INFO:tasks.workunit.client.0.smithi057.stderr:+ wait 2024-04-25T23:51:59.478 INFO:tasks.workunit.client.0.smithi057.stderr:+ sqlite3 -cmd '.output /dev/null' -cmd '.load libcephsqlite.so' -cmd 'pragma journal_mode = PERSIST' -cmd '.open file:///cephsqlite:test_libcephsqlite.sh/baz.db?vfs=ceph' -cmd '.output stdout' 2024-04-25T23:52:00.660 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:00.660 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:00.660 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:00.661 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:00.661 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:00.661 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:00.661 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:00.661 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:00.661 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:00.661 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:00.903 INFO:tasks.workunit.client.0.smithi057.stderr: 2024-04-25T23:52:00.903 INFO:tasks.workunit.client.0.smithi057.stderr:real 0m1.425s 2024-04-25T23:52:00.903 INFO:tasks.workunit.client.0.smithi057.stderr:user 0m0.161s 2024-04-25T23:52:00.903 INFO:tasks.workunit.client.0.smithi057.stderr:sys 0m0.109s 2024-04-25T23:52:00.903 INFO:tasks.workunit.client.0.smithi057.stderr:+ sqlite 2024-04-25T23:52:00.903 INFO:tasks.workunit.client.0.smithi057.stderr:++ repeat 100 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.903 INFO:tasks.workunit.client.0.smithi057.stderr:++ n=100 2024-04-25T23:52:00.904 INFO:tasks.workunit.client.0.smithi057.stderr:++ shift 2024-04-25T23:52:00.904 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i = 0 )) 2024-04-25T23:52:00.904 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.904 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.904 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.904 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.904 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.904 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.904 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.904 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.904 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.905 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.905 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.905 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.905 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.905 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.905 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.905 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.905 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.905 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.905 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.905 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.905 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.905 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.905 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.905 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.906 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.906 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.906 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.906 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.906 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.906 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.906 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.906 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.906 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.906 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.906 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.906 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.906 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.907 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.907 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.907 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.907 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.907 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.907 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.907 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.907 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.907 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.907 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.907 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.907 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.907 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.907 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.907 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.907 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.907 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.907 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.908 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.908 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.908 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.908 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.908 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.908 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.908 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.908 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.908 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.908 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.908 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.908 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.908 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.908 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.908 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.909 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.909 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.909 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.909 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.909 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.909 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.909 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.909 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.909 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.909 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.909 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.909 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.909 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.909 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.909 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.909 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.909 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.910 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.910 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.910 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.910 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.910 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.910 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.910 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.910 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.910 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.910 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.910 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.910 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.910 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.910 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.910 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.911 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.911 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.911 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.911 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.911 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.911 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.911 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.911 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.911 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.911 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.911 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.911 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.911 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.911 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.912 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.912 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.912 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.912 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.912 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.912 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.912 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.912 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.912 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.912 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.912 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.912 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.912 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.913 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.913 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.913 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.913 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.913 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.913 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.913 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.913 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.913 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.913 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.913 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.913 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.913 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.913 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.914 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.914 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.914 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.914 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.914 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.914 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.914 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.914 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.914 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.914 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.914 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.915 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.915 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.915 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.915 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.915 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.915 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.915 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.915 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.915 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.915 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.915 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.915 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.915 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.916 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.916 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.916 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.916 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.916 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.916 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.916 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.916 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.916 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.916 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.916 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.916 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.916 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.916 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.916 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.916 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.917 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.917 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.917 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.917 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.917 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.917 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.917 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.917 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.917 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.917 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.917 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.917 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.917 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.917 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.917 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.918 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.918 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.918 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.918 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.918 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.918 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.918 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.918 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.918 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.918 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.918 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.918 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.918 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.918 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.919 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.919 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.919 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.919 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.919 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.919 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.919 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.919 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.919 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.919 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.919 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.919 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.919 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.919 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.919 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.919 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.920 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.920 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.920 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.920 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.920 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.920 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.920 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.920 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.920 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.920 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.920 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.920 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.920 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.920 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.920 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.921 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.921 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.921 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.921 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.921 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.921 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.921 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.921 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.921 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.921 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.921 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.921 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.921 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.921 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.921 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.922 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.922 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.922 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.922 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.922 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.922 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.922 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.922 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.922 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.922 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.922 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.922 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.922 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.922 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.923 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.923 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.923 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.923 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.923 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.923 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.923 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.923 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.923 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.923 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.923 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.923 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.923 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.923 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.923 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.924 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.924 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.924 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.924 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.924 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.924 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.924 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.924 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.924 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.924 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.924 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.924 INFO:tasks.workunit.client.0.smithi057.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-04-25T23:52:00.924 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( ++i )) 2024-04-25T23:52:00.924 INFO:tasks.workunit.client.0.smithi057.stderr:++ (( i < 100 )) 2024-04-25T23:52:00.924 INFO:tasks.workunit.client.0.smithi057.stderr:+ background= 2024-04-25T23:52:00.925 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' '' = b ']' 2024-04-25T23:52:00.925 INFO:tasks.workunit.client.0.smithi057.stderr:++ cat 2024-04-25T23:52:00.925 INFO:tasks.workunit.client.0.smithi057.stderr:+ a='BEGIN TRANSACTION; 2024-04-25T23:52:00.925 INFO:tasks.workunit.client.0.smithi057.stderr:CREATE TABLE IF NOT EXISTS rand(text BLOB NOT NULL); 2024-04-25T23:52:00.925 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.925 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.925 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.925 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.925 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.925 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.925 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.925 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.925 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.925 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.925 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.926 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.926 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.926 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.926 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.926 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.926 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.926 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.926 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.926 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.926 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.926 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.926 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.926 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.926 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.926 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.927 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.927 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.927 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.927 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.927 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.927 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.927 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.927 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.927 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.927 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.927 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.927 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.927 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.927 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.927 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.928 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.928 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.928 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.928 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.928 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.928 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.928 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.928 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.928 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.930 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.930 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.930 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.930 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.931 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.931 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.931 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.931 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.931 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.931 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.931 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.931 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.931 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.931 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.931 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.931 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.931 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.931 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.931 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.932 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.932 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.932 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.932 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.932 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.932 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.932 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.932 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.932 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.932 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.932 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.932 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.933 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.933 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.933 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.933 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.933 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.933 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.933 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.933 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.933 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.933 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.933 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.933 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.933 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.934 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.934 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.934 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.934 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.934 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.934 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.934 INFO:tasks.workunit.client.0.smithi057.stderr:COMMIT; 2024-04-25T23:52:00.934 INFO:tasks.workunit.client.0.smithi057.stderr:SELECT LENGTH(text) FROM rand; 2024-04-25T23:52:00.934 INFO:tasks.workunit.client.0.smithi057.stderr:DROP TABLE rand;' 2024-04-25T23:52:00.934 INFO:tasks.workunit.client.0.smithi057.stderr:+ printf %s 'BEGIN TRANSACTION; 2024-04-25T23:52:00.934 INFO:tasks.workunit.client.0.smithi057.stderr:CREATE TABLE IF NOT EXISTS rand(text BLOB NOT NULL); 2024-04-25T23:52:00.934 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.934 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.934 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.935 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.935 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.935 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.935 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.935 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.935 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.935 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.935 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.935 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.935 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.935 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.935 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.935 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.935 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.935 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.935 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.936 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.936 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.936 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.936 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.936 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.936 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.936 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.936 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.936 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.936 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.936 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.936 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.936 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.936 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.936 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.937 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.937 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.937 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.937 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.937 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.937 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.937 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.937 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.937 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.937 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.937 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.937 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.937 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.937 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.938 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.938 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.938 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.938 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.938 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.938 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.938 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.938 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.938 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.938 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.938 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.938 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.938 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.938 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.939 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.939 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.939 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.939 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.939 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.939 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.939 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.939 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.939 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.939 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.939 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.939 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.939 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.939 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.939 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.940 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.940 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.940 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.940 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.940 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.940 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.940 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.940 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.940 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.940 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.940 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.940 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.940 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.940 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.940 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.941 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.941 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.941 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.941 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.941 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.941 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.941 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.941 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.941 INFO:tasks.workunit.client.0.smithi057.stderr:COMMIT; 2024-04-25T23:52:00.941 INFO:tasks.workunit.client.0.smithi057.stderr:SELECT LENGTH(text) FROM rand; 2024-04-25T23:52:00.941 INFO:tasks.workunit.client.0.smithi057.stderr:DROP TABLE rand;' 2024-04-25T23:52:00.941 INFO:tasks.workunit.client.0.smithi057.stderr:BEGIN TRANSACTION; 2024-04-25T23:52:00.941 INFO:tasks.workunit.client.0.smithi057.stderr:CREATE TABLE IF NOT EXISTS rand(text BLOB NOT NULL); 2024-04-25T23:52:00.941 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.941 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.942 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.942 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.942 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.942 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.942 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.942 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.942 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.942 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.942 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.942 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.942 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.942 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.942 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.942 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.942 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.943 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.943 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.943 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.943 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.943 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.943 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.943 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.943 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.943 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.943 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.943 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.943 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.943 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.943 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.943 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.943 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.944 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.944 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.944 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.944 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.944 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.944 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.944 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.944 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.944 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.944 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.944 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.944 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.944 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.944 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.944 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.945 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.945 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.945 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.945 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.945 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.945 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.945 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.945 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.945 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.945 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.945 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.945 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.945 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.945 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.945 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.946 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.946 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.946 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.946 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.946 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.946 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.946 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.946 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.946 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.946 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.946 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.946 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.946 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.946 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.947 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.947 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.947 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.947 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.947 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.947 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.947 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.947 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.947 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.947 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.947 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.947 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.947 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.947 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.947 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.948 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.948 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.948 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.948 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.948 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.948 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.948 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.948 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-04-25T23:52:00.948 INFO:tasks.workunit.client.0.smithi057.stderr:COMMIT; 2024-04-25T23:52:00.948 INFO:tasks.workunit.client.0.smithi057.stderr:SELECT LENGTH(text) FROM rand; 2024-04-25T23:52:00.948 INFO:tasks.workunit.client.0.smithi057.stderr:DROP TABLE rand;+ '[' '' '!=' b ']' 2024-04-25T23:52:00.948 INFO:tasks.workunit.client.0.smithi057.stderr:+ wait 2024-04-25T23:52:00.948 INFO:tasks.workunit.client.0.smithi057.stderr:+ sqlite3 -cmd '.output /dev/null' -cmd '.load libcephsqlite.so' -cmd 'pragma journal_mode = PERSIST' -cmd '.open file:///cephsqlite:test_libcephsqlite.sh/baz.db?vfs=ceph' -cmd '.output stdout' 2024-04-25T23:52:01.419 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.419 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.419 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.419 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.419 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.419 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.419 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.419 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.419 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.419 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.419 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.419 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.419 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.420 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.420 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.420 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.420 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.420 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.420 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.420 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.420 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.420 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.420 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.420 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.420 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.420 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.420 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.420 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.421 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.421 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.421 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.421 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.421 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.421 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.421 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.421 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.421 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.421 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.421 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.421 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.421 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.421 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.421 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.421 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.422 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.422 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.422 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.422 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.422 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.422 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.422 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.422 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.422 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.422 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.422 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.422 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.422 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.422 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.422 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.423 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.423 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.423 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.423 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.423 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.423 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.423 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.423 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.423 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.423 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.423 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.423 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.423 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.423 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.423 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.423 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.424 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.424 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.424 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.424 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.424 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.424 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.424 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.424 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.424 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.424 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.424 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.424 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.424 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.424 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.424 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.424 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.425 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.425 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.425 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.425 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.425 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.425 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.425 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.425 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:01.425 INFO:tasks.workunit.client.0.smithi057.stdout:4096 2024-04-25T23:52:02.564 INFO:tasks.workunit.client.0.smithi057.stderr: 2024-04-25T23:52:02.564 INFO:tasks.workunit.client.0.smithi057.stderr:real 0m1.668s 2024-04-25T23:52:02.564 INFO:tasks.workunit.client.0.smithi057.stderr:user 0m0.219s 2024-04-25T23:52:02.564 INFO:tasks.workunit.client.0.smithi057.stderr:sys 0m0.090s 2024-04-25T23:52:02.565 INFO:tasks.workunit.client.0.smithi057.stderr:+ striper rm baz.db 2024-04-25T23:52:02.565 INFO:tasks.workunit.client.0.smithi057.stderr:+ rados --pool=cephsqlite --namespace=test_libcephsqlite.sh --striper rm baz.db 2024-04-25T23:52:02.622 INFO:tasks.workunit.client.0.smithi057.stderr:+ date 2024-04-25T23:52:02.622 INFO:tasks.workunit.client.0.smithi057.stdout:Thu Apr 25 11:52:02 PM UTC 2024 2024-04-25T23:52:02.623 INFO:tasks.workunit.client.0.smithi057.stderr:+ sqlite b 2024-04-25T23:52:02.623 INFO:tasks.workunit.client.0.smithi057.stderr:+ background=b 2024-04-25T23:52:02.623 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' b = b ']' 2024-04-25T23:52:02.623 INFO:tasks.workunit.client.0.smithi057.stderr:+ shift 2024-04-25T23:52:02.623 INFO:tasks.workunit.client.0.smithi057.stderr:++ cat 2024-04-25T23:52:02.624 INFO:tasks.workunit.client.0.smithi057.stderr:+ a='CREATE TABLE foo (a BLOB); 2024-04-25T23:52:02.624 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO foo VALUES ("start"); 2024-04-25T23:52:02.624 INFO:tasks.workunit.client.0.smithi057.stderr:WITH RECURSIVE c(x) AS 2024-04-25T23:52:02.625 INFO:tasks.workunit.client.0.smithi057.stderr: ( 2024-04-25T23:52:02.625 INFO:tasks.workunit.client.0.smithi057.stderr: VALUES(1) 2024-04-25T23:52:02.625 INFO:tasks.workunit.client.0.smithi057.stderr: UNION ALL 2024-04-25T23:52:02.625 INFO:tasks.workunit.client.0.smithi057.stderr: SELECT x+1 2024-04-25T23:52:02.625 INFO:tasks.workunit.client.0.smithi057.stderr: FROM c 2024-04-25T23:52:02.625 INFO:tasks.workunit.client.0.smithi057.stderr: ) 2024-04-25T23:52:02.625 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO foo (a) 2024-04-25T23:52:02.625 INFO:tasks.workunit.client.0.smithi057.stderr: SELECT RANDOMBLOB(1<<20) 2024-04-25T23:52:02.625 INFO:tasks.workunit.client.0.smithi057.stderr: FROM c 2024-04-25T23:52:02.625 INFO:tasks.workunit.client.0.smithi057.stderr: LIMIT (1<<20);' 2024-04-25T23:52:02.625 INFO:tasks.workunit.client.0.smithi057.stderr:+ printf %s 'CREATE TABLE foo (a BLOB); 2024-04-25T23:52:02.625 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO foo VALUES ("start"); 2024-04-25T23:52:02.625 INFO:tasks.workunit.client.0.smithi057.stderr:WITH RECURSIVE c(x) AS 2024-04-25T23:52:02.625 INFO:tasks.workunit.client.0.smithi057.stderr: ( 2024-04-25T23:52:02.626 INFO:tasks.workunit.client.0.smithi057.stderr: VALUES(1) 2024-04-25T23:52:02.626 INFO:tasks.workunit.client.0.smithi057.stderr: UNION ALL 2024-04-25T23:52:02.626 INFO:tasks.workunit.client.0.smithi057.stderr: SELECT x+1 2024-04-25T23:52:02.626 INFO:tasks.workunit.client.0.smithi057.stderr: FROM c 2024-04-25T23:52:02.626 INFO:tasks.workunit.client.0.smithi057.stderr: ) 2024-04-25T23:52:02.626 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO foo (a) 2024-04-25T23:52:02.626 INFO:tasks.workunit.client.0.smithi057.stderr: SELECT RANDOMBLOB(1<<20) 2024-04-25T23:52:02.626 INFO:tasks.workunit.client.0.smithi057.stderr: FROM c 2024-04-25T23:52:02.626 INFO:tasks.workunit.client.0.smithi057.stderr: LIMIT (1<<20);' 2024-04-25T23:52:02.626 INFO:tasks.workunit.client.0.smithi057.stderr:CREATE TABLE foo (a BLOB); 2024-04-25T23:52:02.626 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO foo VALUES ("start"); 2024-04-25T23:52:02.626 INFO:tasks.workunit.client.0.smithi057.stderr:WITH RECURSIVE c(x) AS 2024-04-25T23:52:02.627 INFO:tasks.workunit.client.0.smithi057.stderr: ( 2024-04-25T23:52:02.627 INFO:tasks.workunit.client.0.smithi057.stderr: VALUES(1) 2024-04-25T23:52:02.627 INFO:tasks.workunit.client.0.smithi057.stderr: UNION ALL 2024-04-25T23:52:02.627 INFO:tasks.workunit.client.0.smithi057.stderr: SELECT x+1 2024-04-25T23:52:02.627 INFO:tasks.workunit.client.0.smithi057.stderr: FROM c 2024-04-25T23:52:02.627 INFO:tasks.workunit.client.0.smithi057.stderr: ) 2024-04-25T23:52:02.627 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO foo (a) 2024-04-25T23:52:02.627 INFO:tasks.workunit.client.0.smithi057.stderr: SELECT RANDOMBLOB(1<<20) 2024-04-25T23:52:02.627 INFO:tasks.workunit.client.0.smithi057.stderr: FROM c 2024-04-25T23:52:02.627 INFO:tasks.workunit.client.0.smithi057.stderr: LIMIT (1<<20);+ '[' b '!=' b ']' 2024-04-25T23:52:02.627 INFO:tasks.workunit.client.0.smithi057.stderr:+ sleep 10 2024-04-25T23:52:02.627 INFO:tasks.workunit.client.0.smithi057.stderr:+ sqlite3 -cmd '.output /dev/null' -cmd '.load libcephsqlite.so' -cmd 'pragma journal_mode = PERSIST' -cmd '.open file:///cephsqlite:test_libcephsqlite.sh/baz.db?vfs=ceph' -cmd '.output stdout' 2024-04-25T23:52:12.625 INFO:tasks.workunit.client.0.smithi057.stdout:done 2024-04-25T23:52:12.626 INFO:tasks.workunit.client.0.smithi057.stderr:+ echo done 2024-04-25T23:52:12.626 INFO:tasks.workunit.client.0.smithi057.stderr:+ jobs -l 2024-04-25T23:52:12.626 INFO:tasks.workunit.client.0.smithi057.stdout:[1]+ 29875 Running sqlite3 -cmd '.output /dev/null' -cmd '.load libcephsqlite.so' -cmd 'pragma journal_mode = PERSIST' -cmd ".open file:///$pool:$ns/baz.db?vfs=ceph" -cmd '.output stdout' <<< "$a" & 2024-04-25T23:52:12.627 INFO:tasks.workunit.client.0.smithi057.stderr:++ jobs -p 2024-04-25T23:52:12.627 INFO:tasks.workunit.client.0.smithi057.stderr:+ kill -KILL -- 29875 2024-04-25T23:52:12.627 INFO:tasks.workunit.client.0.smithi057.stderr:+ date 2024-04-25T23:52:12.627 INFO:tasks.workunit.client.0.smithi057.stdout:Thu Apr 25 11:52:12 PM UTC 2024 2024-04-25T23:52:12.628 INFO:tasks.workunit.client.0.smithi057.stderr:+ wait 2024-04-25T23:52:12.630 INFO:tasks.workunit.client.0.smithi057.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_libcephsqlite.sh: line 95: 29875 Killed sqlite3 -cmd '.output /dev/null' -cmd '.load libcephsqlite.so' -cmd 'pragma journal_mode = PERSIST' -cmd ".open file:///$pool:$ns/baz.db?vfs=ceph" -cmd '.output stdout' <<< "$a" 2024-04-25T23:52:12.630 INFO:tasks.workunit.client.0.smithi057.stderr:+ date 2024-04-25T23:52:12.631 INFO:tasks.workunit.client.0.smithi057.stdout:Thu Apr 25 11:52:12 PM UTC 2024 2024-04-25T23:52:12.632 INFO:tasks.workunit.client.0.smithi057.stderr:++ sqlite 2024-04-25T23:52:12.632 INFO:tasks.workunit.client.0.smithi057.stderr:++ background= 2024-04-25T23:52:12.632 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' = b ']' 2024-04-25T23:52:12.633 INFO:tasks.workunit.client.0.smithi057.stderr:+++ cat 2024-04-25T23:52:12.633 INFO:tasks.workunit.client.0.smithi057.stderr:++ a='SELECT COUNT(*) FROM foo;' 2024-04-25T23:52:12.633 INFO:tasks.workunit.client.0.smithi057.stderr:++ printf %s 'SELECT COUNT(*) FROM foo;' 2024-04-25T23:52:12.633 INFO:tasks.workunit.client.0.smithi057.stderr:SELECT COUNT(*) FROM foo;++ '[' '' '!=' b ']' 2024-04-25T23:52:12.633 INFO:tasks.workunit.client.0.smithi057.stderr:++ wait 2024-04-25T23:52:12.634 INFO:tasks.workunit.client.0.smithi057.stderr:++ sqlite3 -cmd '.output /dev/null' -cmd '.load libcephsqlite.so' -cmd 'pragma journal_mode = PERSIST' -cmd '.open file:///cephsqlite:test_libcephsqlite.sh/baz.db?vfs=ceph' -cmd '.output stdout' 2024-04-25T23:52:42.431 INFO:tasks.workunit.client.0.smithi057.stderr:+ n=1 2024-04-25T23:52:42.432 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' 1 -eq 1 ']' 2024-04-25T23:52:42.432 INFO:tasks.workunit.client.0.smithi057.stderr:+ striper rm baz.db 2024-04-25T23:52:42.432 INFO:tasks.workunit.client.0.smithi057.stderr:+ rados --pool=cephsqlite --namespace=test_libcephsqlite.sh --striper rm baz.db 2024-04-25T23:52:42.510 INFO:tasks.workunit.client.0.smithi057.stderr:+ date 2024-04-25T23:52:42.514 INFO:tasks.workunit.client.0.smithi057.stdout:Thu Apr 25 11:52:42 PM UTC 2024 2024-04-25T23:52:42.515 INFO:tasks.workunit.client.0.smithi057.stderr:+ sqlite b 2024-04-25T23:52:42.515 INFO:tasks.workunit.client.0.smithi057.stderr:+ background=b 2024-04-25T23:52:42.515 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' b = b ']' 2024-04-25T23:52:42.516 INFO:tasks.workunit.client.0.smithi057.stderr:+ shift 2024-04-25T23:52:42.516 INFO:tasks.workunit.client.0.smithi057.stderr:++ cat 2024-04-25T23:52:42.516 INFO:tasks.workunit.client.0.smithi057.stderr:+ a='CREATE TABLE foo (a BLOB); 2024-04-25T23:52:42.516 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO foo VALUES ("start"); 2024-04-25T23:52:42.516 INFO:tasks.workunit.client.0.smithi057.stderr:WITH RECURSIVE c(x) AS 2024-04-25T23:52:42.516 INFO:tasks.workunit.client.0.smithi057.stderr: ( 2024-04-25T23:52:42.516 INFO:tasks.workunit.client.0.smithi057.stderr: VALUES(1) 2024-04-25T23:52:42.516 INFO:tasks.workunit.client.0.smithi057.stderr: UNION ALL 2024-04-25T23:52:42.516 INFO:tasks.workunit.client.0.smithi057.stderr: SELECT x+1 2024-04-25T23:52:42.516 INFO:tasks.workunit.client.0.smithi057.stderr: FROM c 2024-04-25T23:52:42.516 INFO:tasks.workunit.client.0.smithi057.stderr: ) 2024-04-25T23:52:42.516 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO foo (a) 2024-04-25T23:52:42.516 INFO:tasks.workunit.client.0.smithi057.stderr: SELECT RANDOMBLOB(1<<20) 2024-04-25T23:52:42.516 INFO:tasks.workunit.client.0.smithi057.stderr: FROM c 2024-04-25T23:52:42.516 INFO:tasks.workunit.client.0.smithi057.stderr: LIMIT (1<<20);' 2024-04-25T23:52:42.517 INFO:tasks.workunit.client.0.smithi057.stderr:+ printf %s 'CREATE TABLE foo (a BLOB); 2024-04-25T23:52:42.517 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO foo VALUES ("start"); 2024-04-25T23:52:42.517 INFO:tasks.workunit.client.0.smithi057.stderr:WITH RECURSIVE c(x) AS 2024-04-25T23:52:42.517 INFO:tasks.workunit.client.0.smithi057.stderr: ( 2024-04-25T23:52:42.517 INFO:tasks.workunit.client.0.smithi057.stderr: VALUES(1) 2024-04-25T23:52:42.517 INFO:tasks.workunit.client.0.smithi057.stderr: UNION ALL 2024-04-25T23:52:42.517 INFO:tasks.workunit.client.0.smithi057.stderr: SELECT x+1 2024-04-25T23:52:42.517 INFO:tasks.workunit.client.0.smithi057.stderr: FROM c 2024-04-25T23:52:42.517 INFO:tasks.workunit.client.0.smithi057.stderr: ) 2024-04-25T23:52:42.517 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO foo (a) 2024-04-25T23:52:42.517 INFO:tasks.workunit.client.0.smithi057.stderr: SELECT RANDOMBLOB(1<<20) 2024-04-25T23:52:42.517 INFO:tasks.workunit.client.0.smithi057.stderr: FROM c 2024-04-25T23:52:42.517 INFO:tasks.workunit.client.0.smithi057.stderr: LIMIT (1<<20);' 2024-04-25T23:52:42.517 INFO:tasks.workunit.client.0.smithi057.stderr:CREATE TABLE foo (a BLOB); 2024-04-25T23:52:42.517 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO foo VALUES ("start"); 2024-04-25T23:52:42.517 INFO:tasks.workunit.client.0.smithi057.stderr:WITH RECURSIVE c(x) AS 2024-04-25T23:52:42.518 INFO:tasks.workunit.client.0.smithi057.stderr: ( 2024-04-25T23:52:42.518 INFO:tasks.workunit.client.0.smithi057.stderr: VALUES(1) 2024-04-25T23:52:42.518 INFO:tasks.workunit.client.0.smithi057.stderr: UNION ALL 2024-04-25T23:52:42.518 INFO:tasks.workunit.client.0.smithi057.stderr: SELECT x+1 2024-04-25T23:52:42.518 INFO:tasks.workunit.client.0.smithi057.stderr: FROM c 2024-04-25T23:52:42.518 INFO:tasks.workunit.client.0.smithi057.stderr: ) 2024-04-25T23:52:42.518 INFO:tasks.workunit.client.0.smithi057.stderr:INSERT INTO foo (a) 2024-04-25T23:52:42.518 INFO:tasks.workunit.client.0.smithi057.stderr: SELECT RANDOMBLOB(1<<20) 2024-04-25T23:52:42.518 INFO:tasks.workunit.client.0.smithi057.stderr: FROM c 2024-04-25T23:52:42.518 INFO:tasks.workunit.client.0.smithi057.stderr: LIMIT (1<<20);+ '[' b '!=' b ']' 2024-04-25T23:52:42.518 INFO:tasks.workunit.client.0.smithi057.stderr:+ sleep 20 2024-04-25T23:52:42.518 INFO:tasks.workunit.client.0.smithi057.stderr:+ sqlite3 -cmd '.output /dev/null' -cmd '.load libcephsqlite.so' -cmd 'pragma journal_mode = PERSIST' -cmd '.open file:///cephsqlite:test_libcephsqlite.sh/baz.db?vfs=ceph' -cmd '.output stdout' 2024-04-25T23:53:02.514 INFO:tasks.workunit.client.0.smithi057.stdout:[1]+ 29943 Running sqlite3 -cmd '.output /dev/null' -cmd '.load libcephsqlite.so' -cmd 'pragma journal_mode = PERSIST' -cmd ".open file:///$pool:$ns/baz.db?vfs=ceph" -cmd '.output stdout' <<< "$a" & 2024-04-25T23:53:02.515 INFO:tasks.workunit.client.0.smithi057.stderr:+ jobs -l 2024-04-25T23:53:02.515 INFO:tasks.workunit.client.0.smithi057.stderr:++ jobs -p 2024-04-25T23:53:02.515 INFO:tasks.workunit.client.0.smithi057.stderr:+ kill -STOP -- 29943 2024-04-25T23:53:02.515 INFO:tasks.workunit.client.0.smithi057.stderr:+ sleep 45 2024-04-25T23:53:47.516 INFO:tasks.workunit.client.0.smithi057.stderr:+ date 2024-04-25T23:53:47.517 INFO:tasks.workunit.client.0.smithi057.stdout:Thu Apr 25 11:53:47 PM UTC 2024 2024-04-25T23:53:47.518 INFO:tasks.workunit.client.0.smithi057.stderr:++ jobs -p 2024-04-25T23:53:47.518 INFO:tasks.workunit.client.0.smithi057.stderr:+ kill -CONT -- 29943 2024-04-25T23:53:47.518 INFO:tasks.workunit.client.0.smithi057.stderr:+ sleep 10 2024-04-25T23:53:47.610 INFO:tasks.workunit.client.0.smithi057.stderr:Error: near line 3: stepping, disk I/O error (10) 2024-04-25T23:53:57.520 INFO:tasks.workunit.client.0.smithi057.stderr:+ date 2024-04-25T23:53:57.521 INFO:tasks.workunit.client.0.smithi057.stdout:Thu Apr 25 11:53:57 PM UTC 2024 2024-04-25T23:53:57.522 INFO:tasks.workunit.client.0.smithi057.stderr:+ wait 2024-04-25T23:53:57.522 INFO:tasks.workunit.client.0.smithi057.stderr:+ date 2024-04-25T23:53:57.523 INFO:tasks.workunit.client.0.smithi057.stdout:Thu Apr 25 11:53:57 PM UTC 2024 2024-04-25T23:53:57.524 INFO:tasks.workunit.client.0.smithi057.stderr:++ sqlite 2024-04-25T23:53:57.524 INFO:tasks.workunit.client.0.smithi057.stderr:++ background= 2024-04-25T23:53:57.524 INFO:tasks.workunit.client.0.smithi057.stderr:++ '[' '' = b ']' 2024-04-25T23:53:57.524 INFO:tasks.workunit.client.0.smithi057.stderr:+++ cat 2024-04-25T23:53:57.526 INFO:tasks.workunit.client.0.smithi057.stderr:++ a='SELECT COUNT(*) FROM foo;' 2024-04-25T23:53:57.526 INFO:tasks.workunit.client.0.smithi057.stderr:++ printf %s 'SELECT COUNT(*) FROM foo;' 2024-04-25T23:53:57.526 INFO:tasks.workunit.client.0.smithi057.stderr:SELECT COUNT(*) FROM foo;++ '[' '' '!=' b ']' 2024-04-25T23:53:57.526 INFO:tasks.workunit.client.0.smithi057.stderr:++ wait 2024-04-25T23:53:57.526 INFO:tasks.workunit.client.0.smithi057.stderr:++ sqlite3 -cmd '.output /dev/null' -cmd '.load libcephsqlite.so' -cmd 'pragma journal_mode = PERSIST' -cmd '.open file:///cephsqlite:test_libcephsqlite.sh/baz.db?vfs=ceph' -cmd '.output stdout' 2024-04-25T23:53:58.314 INFO:tasks.workunit.client.0.smithi057.stderr:+ n=1 2024-04-25T23:53:58.314 INFO:tasks.workunit.client.0.smithi057.stderr:+ '[' 1 -eq 1 ']' 2024-04-25T23:53:58.314 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-25T23:53:58.315 DEBUG:teuthology.orchestra.run.smithi057:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-25T23:53:58.369 INFO:tasks.workunit:Stopping ['rados/test_libcephsqlite.sh cephsqlite'] on client.0... 2024-04-25T23:53:58.369 DEBUG:teuthology.orchestra.run.smithi057:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-25T23:53:58.733 DEBUG:teuthology.parallel:result is None 2024-04-25T23:53:58.734 DEBUG:teuthology.orchestra.run.smithi057:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-25T23:53:58.747 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-25T23:53:58.747 DEBUG:teuthology.orchestra.run.smithi057:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-04-25T23:53:58.807 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-04-25T23:53:58.807 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-25T23:53:58.824 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T23:53:58.824 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T23:53:58.983 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:58.996+0000 7ff63dabe640 1 Processor -- start 2024-04-25T23:53:58.983 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:58.996+0000 7ff63dabe640 1 -- start start 2024-04-25T23:53:58.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.000+0000 7ff63dabe640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7ff638071d50 0x7ff638070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:53:58.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.000+0000 7ff63dabe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff63806ea90 0x7ff638142030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:53:58.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.000+0000 7ff63dabe640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff638072130 con 0x7ff638142570 2024-04-25T23:53:58.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.000+0000 7ff63dabe640 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 -- 0x7ff6380adb10 con 0x7ff638071d50 2024-04-25T23:53:58.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.000+0000 7ff63dabe640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7ff6381461e0 con 0x7ff63806ea90 2024-04-25T23:53:58.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.000+0000 7ff6377fe640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7ff638071d50 0x7ff638070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:53:58.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.000+0000 7ff6377fe640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7ff638071d50 0x7ff638070ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37116/0 (socket says 172.21.15.57:37116) 2024-04-25T23:53:58.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.000+0000 7ff6377fe640 1 -- 172.21.15.57:0/1952155159 learned_addr learned my addr 172.21.15.57:0/1952155159 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:53:58.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.000+0000 7ff636ffd640 1 --2- 172.21.15.57:0/1952155159 >> v2:172.21.15.80:3300/0 conn(0x7ff63806ea90 0x7ff638142030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:53:58.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.000+0000 7ff6377fe640 1 -- 172.21.15.57:0/1952155159 >> v2:172.21.15.80:3300/0 conn(0x7ff63806ea90 msgr2=0x7ff638142030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:53:58.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.000+0000 7ff6377fe640 1 --2- 172.21.15.57:0/1952155159 >> v2:172.21.15.80:3300/0 conn(0x7ff63806ea90 0x7ff638142030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:53:58.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.000+0000 7ff6377fe640 1 -- 172.21.15.57:0/1952155159 >> v1:172.21.15.80:6789/0 conn(0x7ff638142570 legacy=0x7ff638146670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:53:58.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.000+0000 7ff6377fe640 1 -- 172.21.15.57:0/1952155159 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6381429a0 con 0x7ff638071d50 2024-04-25T23:53:58.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.000+0000 7ff6377fe640 1 --2- 172.21.15.57:0/1952155159 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff638071d50 0x7ff638070ff0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff63806c720 tx=0x7ff62001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71d1a0432b9c595c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:53:58.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.000+0000 7ff6367fc640 1 -- 172.21.15.57:0/1952155159 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6200aa650 con 0x7ff638071d50 2024-04-25T23:53:58.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.000+0000 7ff6367fc640 1 -- 172.21.15.57:0/1952155159 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6200af070 con 0x7ff638071d50 2024-04-25T23:53:58.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.000+0000 7ff6367fc640 1 -- 172.21.15.57:0/1952155159 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6200aaa10 con 0x7ff638071d50 2024-04-25T23:53:58.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.004+0000 7ff63dabe640 1 -- 172.21.15.57:0/1952155159 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff638071d50 msgr2=0x7ff638070ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:53:58.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.004+0000 7ff63dabe640 1 --2- 172.21.15.57:0/1952155159 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff638071d50 0x7ff638070ff0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff63806c720 tx=0x7ff62001c4d0 comp rx=0 tx=0).stop 2024-04-25T23:53:58.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.004+0000 7ff63dabe640 1 -- 172.21.15.57:0/1952155159 shutdown_connections 2024-04-25T23:53:58.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.004+0000 7ff63dabe640 1 --2- 172.21.15.57:0/1952155159 >> v2:172.21.15.80:3300/0 conn(0x7ff63806ea90 0x7ff638142030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:53:58.987 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.004+0000 7ff63dabe640 1 --2- 172.21.15.57:0/1952155159 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff638071d50 0x7ff638070ff0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:53:58.988 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.004+0000 7ff63dabe640 1 -- 172.21.15.57:0/1952155159 >> 172.21.15.57:0/1952155159 conn(0x7ff6380b9110 msgr2=0x7ff6380b9520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:53:58.988 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.004+0000 7ff63dabe640 1 -- 172.21.15.57:0/1952155159 shutdown_connections 2024-04-25T23:53:58.988 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.004+0000 7ff63dabe640 1 -- 172.21.15.57:0/1952155159 wait complete. 2024-04-25T23:53:58.988 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.004+0000 7ff63dabe640 1 Processor -- start 2024-04-25T23:53:58.988 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.004+0000 7ff63dabe640 1 -- start start 2024-04-25T23:53:58.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.004+0000 7ff63dabe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff63806ea90 0x7ff6381de7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:53:58.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.004+0000 7ff63dabe640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff638071d50 0x7ff6381ded00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:53:58.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.004+0000 7ff63dabe640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7ff638142570 0x7ff638136a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:53:58.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.004+0000 7ff63dabe640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7ff638150110 con 0x7ff638071d50 2024-04-25T23:53:58.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.004+0000 7ff63dabe640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff63814ff90 con 0x7ff63806ea90 2024-04-25T23:53:58.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.004+0000 7ff63dabe640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7ff638150290 con 0x7ff638142570 2024-04-25T23:53:58.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.004+0000 7ff636ffd640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff638071d50 0x7ff6381ded00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:53:58.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.004+0000 7ff637fff640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7ff638142570 0x7ff638136a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:53:58.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.004+0000 7ff636ffd640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff638071d50 0x7ff6381ded00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37122/0 (socket says 172.21.15.57:37122) 2024-04-25T23:53:58.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.004+0000 7ff636ffd640 1 -- 172.21.15.57:0/3343227223 learned_addr learned my addr 172.21.15.57:0/3343227223 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:53:58.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.004+0000 7ff6377fe640 1 --2- 172.21.15.57:0/3343227223 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff63806ea90 0x7ff6381de7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:53:58.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.004+0000 7ff637fff640 1 -- 172.21.15.57:0/3343227223 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff63806ea90 msgr2=0x7ff6381de7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:53:58.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.004+0000 7ff637fff640 1 --2- 172.21.15.57:0/3343227223 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff63806ea90 0x7ff6381de7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:53:58.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.004+0000 7ff637fff640 1 -- 172.21.15.57:0/3343227223 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff638071d50 msgr2=0x7ff6381ded00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:53:58.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.004+0000 7ff637fff640 1 --2- 172.21.15.57:0/3343227223 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff638071d50 0x7ff6381ded00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:53:58.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.004+0000 7ff637fff640 1 -- 172.21.15.57:0/3343227223 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff63813ec30 con 0x7ff638142570 2024-04-25T23:53:58.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.004+0000 7ff636ffd640 1 --2- 172.21.15.57:0/3343227223 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff638071d50 0x7ff6381ded00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T23:53:58.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.004+0000 7ff637fff640 1 --2- 172.21.15.57:0/3343227223 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7ff638142570 0x7ff638136a30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff63000c730 tx=0x7ff63000cbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:53:58.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.004+0000 7ff62ffff640 1 -- 172.21.15.57:0/3343227223 <== mon.2 v2:172.21.15.57:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff630003b80 con 0x7ff638142570 2024-04-25T23:53:58.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.004+0000 7ff62ffff640 1 -- 172.21.15.57:0/3343227223 <== mon.2 v2:172.21.15.57:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff630003d20 con 0x7ff638142570 2024-04-25T23:53:58.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.004+0000 7ff63dabe640 1 -- 172.21.15.57:0/3343227223 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6381376f0 con 0x7ff638142570 2024-04-25T23:53:58.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.004+0000 7ff62ffff640 1 -- 172.21.15.57:0/3343227223 <== mon.2 v2:172.21.15.57:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff630005310 con 0x7ff638142570 2024-04-25T23:53:58.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.004+0000 7ff63dabe640 1 -- 172.21.15.57:0/3343227223 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff638137140 con 0x7ff638142570 2024-04-25T23:53:58.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.008+0000 7ff63dabe640 1 -- 172.21.15.57:0/3343227223 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6380761e0 con 0x7ff638142570 2024-04-25T23:53:58.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.008+0000 7ff62ffff640 1 -- 172.21.15.57:0/3343227223 <== mon.2 v2:172.21.15.57:3301/0 4 ==== mgrmap(e 5) ==== 87410+0+0 (secure 0 0 0) 0x7ff6300056c0 con 0x7ff638142570 2024-04-25T23:53:58.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.008+0000 7ff62ffff640 1 --2- 172.21.15.57:0/3343227223 >> v2:172.21.15.80:6816/2493457677 conn(0x7ff6180678b0 0x7ff618069d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:53:58.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.008+0000 7ff62ffff640 1 -- 172.21.15.57:0/3343227223 <== mon.2 v2:172.21.15.57:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 4525+0+0 (secure 0 0 0) 0x7ff630085740 con 0x7ff638142570 2024-04-25T23:53:58.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.008+0000 7ff6377fe640 1 --2- 172.21.15.57:0/3343227223 >> v2:172.21.15.80:6816/2493457677 conn(0x7ff6180678b0 0x7ff618069d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:53:58.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.008+0000 7ff6377fe640 1 --2- 172.21.15.57:0/3343227223 >> v2:172.21.15.80:6816/2493457677 conn(0x7ff6180678b0 0x7ff618069d70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff6200ae000 tx=0x7ff6200a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:53:58.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.012+0000 7ff62ffff640 1 -- 172.21.15.57:0/3343227223 <== mon.2 v2:172.21.15.57:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff638072130 con 0x7ff638142570 2024-04-25T23:53:59.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.172+0000 7ff63dabe640 1 -- 172.21.15.57:0/3343227223 --> v2:172.21.15.80:6816/2493457677 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff6381df430 con 0x7ff6180678b0 2024-04-25T23:53:59.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.180+0000 7ff62ffff640 1 -- 172.21.15.57:0/3343227223 <== mgr.4100 v2:172.21.15.80:6816/2493457677 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+117689 (secure 0 0 0) 0x7ff6381df430 con 0x7ff6180678b0 2024-04-25T23:53:59.168 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-25T23:53:59.169 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2024-04-25T23:53:59.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.184+0000 7ff63dabe640 1 -- 172.21.15.57:0/3343227223 >> v2:172.21.15.80:6816/2493457677 conn(0x7ff6180678b0 msgr2=0x7ff618069d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:53:59.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.184+0000 7ff63dabe640 1 --2- 172.21.15.57:0/3343227223 >> v2:172.21.15.80:6816/2493457677 conn(0x7ff6180678b0 0x7ff618069d70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff6200ae000 tx=0x7ff6200a7000 comp rx=0 tx=0).stop 2024-04-25T23:53:59.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.184+0000 7ff63dabe640 1 -- 172.21.15.57:0/3343227223 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7ff638142570 msgr2=0x7ff638136a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:53:59.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.184+0000 7ff63dabe640 1 --2- 172.21.15.57:0/3343227223 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7ff638142570 0x7ff638136a30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff63000c730 tx=0x7ff63000cbf0 comp rx=0 tx=0).stop 2024-04-25T23:53:59.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.184+0000 7ff63dabe640 1 -- 172.21.15.57:0/3343227223 shutdown_connections 2024-04-25T23:53:59.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.184+0000 7ff63dabe640 1 --2- 172.21.15.57:0/3343227223 >> v2:172.21.15.80:6816/2493457677 conn(0x7ff6180678b0 0x7ff618069d70 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:53:59.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.184+0000 7ff63dabe640 1 --2- 172.21.15.57:0/3343227223 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7ff638142570 0x7ff638136a30 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:53:59.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.184+0000 7ff63dabe640 1 --2- 172.21.15.57:0/3343227223 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff638071d50 0x7ff6381ded00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:53:59.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.184+0000 7ff63dabe640 1 --2- 172.21.15.57:0/3343227223 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff63806ea90 0x7ff6381de7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:53:59.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.184+0000 7ff63dabe640 1 -- 172.21.15.57:0/3343227223 >> 172.21.15.57:0/3343227223 conn(0x7ff6380b9110 msgr2=0x7ff63806ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:53:59.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.184+0000 7ff63dabe640 1 -- 172.21.15.57:0/3343227223 shutdown_connections 2024-04-25T23:53:59.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.184+0000 7ff63dabe640 1 -- 172.21.15.57:0/3343227223 wait complete. 2024-04-25T23:53:59.187 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":166,"stamp":"2024-04-25T23:53:58.919008+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3561436456,"num_objects":879,"num_object_clones":0,"num_object_copies":1758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":879,"num_whiteouts":0,"num_read":2623,"num_read_kb":2618,"num_write":155851,"num_write_kb":5622889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157601,"ondisk_log_size":157601,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":9015800,"kb_used_data":6625464,"kb_used_omap":12,"kb_used_meta":2390259,"kb_avail":745958920,"statfs":{"total":773094113280,"available":763861934080,"internally_reserved":0,"allocated":6784475136,"data_stored":6821918248,"data_compressed":194300696,"data_compressed_allocated":202137600,"data_compressed_original":244916224,"omap_allocated":12751,"internal_metadata":2447625777},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":16384,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":5,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003287"},"pg_stats":[{"pgid":"3.1f","version":"20'4490","reported_seq":4529,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.383532+0000","last_change":"2024-04-25T23:49:04.991824+0000","last_active":"2024-04-25T23:52:42.383532+0000","last_peered":"2024-04-25T23:52:42.383532+0000","last_clean":"2024-04-25T23:52:42.383532+0000","last_became_active":"2024-04-25T23:49:04.991059+0000","last_became_peered":"2024-04-25T23:49:04.991059+0000","last_unstale":"2024-04-25T23:52:42.383532+0000","last_undegraded":"2024-04-25T23:52:42.383532+0000","last_fullsized":"2024-04-25T23:52:42.383532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":4490,"log_dups_size":0,"ondisk_log_size":4490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:06:08.003629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121634816,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":93,"num_read_kb":81,"num_write":4513,"num_write_kb":218308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"21'5438","reported_seq":5465,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:50.093018+0000","last_change":"2024-04-25T23:49:04.991993+0000","last_active":"2024-04-25T23:52:50.093018+0000","last_peered":"2024-04-25T23:52:50.093018+0000","last_clean":"2024-04-25T23:52:50.093018+0000","last_became_active":"2024-04-25T23:49:04.991224+0000","last_became_peered":"2024-04-25T23:49:04.991224+0000","last_unstale":"2024-04-25T23:52:50.093018+0000","last_undegraded":"2024-04-25T23:52:50.093018+0000","last_fullsized":"2024-04-25T23:52:50.093018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":5438,"log_dups_size":0,"ondisk_log_size":5438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:49:45.384039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":45,"num_read_kb":37,"num_write":5439,"num_write_kb":172929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"21'6313","reported_seq":6337,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:56.683805+0000","last_change":"2024-04-25T23:49:04.984448+0000","last_active":"2024-04-25T23:52:56.683805+0000","last_peered":"2024-04-25T23:52:56.683805+0000","last_clean":"2024-04-25T23:52:56.683805+0000","last_became_active":"2024-04-25T23:49:04.982204+0000","last_became_peered":"2024-04-25T23:49:04.982204+0000","last_unstale":"2024-04-25T23:52:56.683805+0000","last_undegraded":"2024-04-25T23:52:56.683805+0000","last_fullsized":"2024-04-25T23:52:56.683805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":6313,"log_dups_size":0,"ondisk_log_size":6313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:23:55.891971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":150994944,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":30,"num_read_kb":85,"num_write":6420,"num_write_kb":204243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"20'3855","reported_seq":3874,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.391433+0000","last_change":"2024-04-25T23:49:04.980119+0000","last_active":"2024-04-25T23:52:42.391433+0000","last_peered":"2024-04-25T23:52:42.391433+0000","last_clean":"2024-04-25T23:52:42.391433+0000","last_became_active":"2024-04-25T23:49:04.979323+0000","last_became_peered":"2024-04-25T23:49:04.979323+0000","last_unstale":"2024-04-25T23:52:42.391433+0000","last_undegraded":"2024-04-25T23:52:42.391433+0000","last_fullsized":"2024-04-25T23:52:42.391433+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":3855,"log_dups_size":0,"ondisk_log_size":3855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:09:46.513536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96535048,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":17,"num_read_kb":12,"num_write":3868,"num_write_kb":155287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"21'4570","reported_seq":4585,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.802622+0000","last_change":"2024-04-25T23:49:05.002350+0000","last_active":"2024-04-25T23:52:42.802622+0000","last_peered":"2024-04-25T23:52:42.802622+0000","last_clean":"2024-04-25T23:52:42.802622+0000","last_became_active":"2024-04-25T23:49:05.001246+0000","last_became_peered":"2024-04-25T23:49:05.001246+0000","last_unstale":"2024-04-25T23:52:42.802622+0000","last_undegraded":"2024-04-25T23:52:42.802622+0000","last_fullsized":"2024-04-25T23:52:42.802622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":4570,"log_dups_size":0,"ondisk_log_size":4570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:27:02.668986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":107675648,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4523,"num_write_kb":223556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"21'6959","reported_seq":6974,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:50.115868+0000","last_change":"2024-04-25T23:49:04.984321+0000","last_active":"2024-04-25T23:52:50.115868+0000","last_peered":"2024-04-25T23:52:50.115868+0000","last_clean":"2024-04-25T23:52:50.115868+0000","last_became_active":"2024-04-25T23:49:04.983518+0000","last_became_peered":"2024-04-25T23:49:04.983518+0000","last_unstale":"2024-04-25T23:52:50.115868+0000","last_undegraded":"2024-04-25T23:52:50.115868+0000","last_fullsized":"2024-04-25T23:52:50.115868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":6959,"log_dups_size":0,"ondisk_log_size":6959,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:31:32.872285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":138412032,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6933,"num_write_kb":206597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"21'5764","reported_seq":5780,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:53:01.763608+0000","last_change":"2024-04-25T23:49:04.985251+0000","last_active":"2024-04-25T23:53:01.763608+0000","last_peered":"2024-04-25T23:53:01.763608+0000","last_clean":"2024-04-25T23:53:01.763608+0000","last_became_active":"2024-04-25T23:49:04.984378+0000","last_became_peered":"2024-04-25T23:49:04.984378+0000","last_unstale":"2024-04-25T23:53:01.763608+0000","last_undegraded":"2024-04-25T23:53:01.763608+0000","last_fullsized":"2024-04-25T23:53:01.763608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":5764,"log_dups_size":0,"ondisk_log_size":5764,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:43:34.630095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121634816,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5717,"num_write_kb":246662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"21'7903","reported_seq":7920,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:53:00.638258+0000","last_change":"2024-04-25T23:49:04.988246+0000","last_active":"2024-04-25T23:53:00.638258+0000","last_peered":"2024-04-25T23:53:00.638258+0000","last_clean":"2024-04-25T23:53:00.638258+0000","last_became_active":"2024-04-25T23:49:04.987468+0000","last_became_peered":"2024-04-25T23:49:04.987468+0000","last_unstale":"2024-04-25T23:53:00.638258+0000","last_undegraded":"2024-04-25T23:53:00.638258+0000","last_fullsized":"2024-04-25T23:53:00.638258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":7903,"log_dups_size":0,"ondisk_log_size":7903,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:21:42.911815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":130023936,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":7908,"num_write_kb":222727,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"21'5984","reported_seq":6008,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:53.277327+0000","last_change":"2024-04-25T23:49:04.989029+0000","last_active":"2024-04-25T23:52:53.277327+0000","last_peered":"2024-04-25T23:52:53.277327+0000","last_clean":"2024-04-25T23:52:53.277327+0000","last_became_active":"2024-04-25T23:49:04.987729+0000","last_became_peered":"2024-04-25T23:49:04.987729+0000","last_unstale":"2024-04-25T23:52:53.277327+0000","last_undegraded":"2024-04-25T23:52:53.277327+0000","last_fullsized":"2024-04-25T23:52:53.277327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":5984,"log_dups_size":0,"ondisk_log_size":5984,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:34:36.865614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121634816,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":5995,"num_write_kb":201733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"21'3557","reported_seq":3881,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.801711+0000","last_change":"2024-04-25T23:49:04.986698+0000","last_active":"2024-04-25T23:52:42.801711+0000","last_peered":"2024-04-25T23:52:42.801711+0000","last_clean":"2024-04-25T23:52:42.801711+0000","last_became_active":"2024-04-25T23:49:04.985957+0000","last_became_peered":"2024-04-25T23:49:04.985957+0000","last_unstale":"2024-04-25T23:52:42.801711+0000","last_undegraded":"2024-04-25T23:52:42.801711+0000","last_fullsized":"2024-04-25T23:52:42.801711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":3557,"log_dups_size":0,"ondisk_log_size":3557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:07:54.597105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":77201936,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":1028,"num_read_kb":921,"num_write":3751,"num_write_kb":130317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"21'7645","reported_seq":7661,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:53:00.534834+0000","last_change":"2024-04-25T23:49:04.992705+0000","last_active":"2024-04-25T23:53:00.534834+0000","last_peered":"2024-04-25T23:53:00.534834+0000","last_clean":"2024-04-25T23:53:00.534834+0000","last_became_active":"2024-04-25T23:49:04.990812+0000","last_became_peered":"2024-04-25T23:49:04.990812+0000","last_unstale":"2024-04-25T23:53:00.534834+0000","last_undegraded":"2024-04-25T23:53:00.534834+0000","last_fullsized":"2024-04-25T23:53:00.534834+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":7645,"log_dups_size":0,"ondisk_log_size":7645,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:27:48.304794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":138412032,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7636,"num_write_kb":198278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"21'5030","reported_seq":5062,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:52.943801+0000","last_change":"2024-04-25T23:49:04.990899+0000","last_active":"2024-04-25T23:52:52.943801+0000","last_peered":"2024-04-25T23:52:52.943801+0000","last_clean":"2024-04-25T23:52:52.943801+0000","last_became_active":"2024-04-25T23:49:04.988250+0000","last_became_peered":"2024-04-25T23:49:04.988250+0000","last_unstale":"2024-04-25T23:52:52.943801+0000","last_undegraded":"2024-04-25T23:52:52.943801+0000","last_fullsized":"2024-04-25T23:52:52.943801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":5030,"log_dups_size":0,"ondisk_log_size":5030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:53:15.377369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":95029072,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":68,"num_read_kb":57,"num_write":5006,"num_write_kb":140314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.385292+0000","last_change":"2024-04-25T23:48:58.911464+0000","last_active":"2024-04-25T23:52:42.385292+0000","last_peered":"2024-04-25T23:52:42.385292+0000","last_clean":"2024-04-25T23:52:42.385292+0000","last_became_active":"2024-04-25T23:48:58.911101+0000","last_became_peered":"2024-04-25T23:48:58.911101+0000","last_unstale":"2024-04-25T23:52:42.385292+0000","last_undegraded":"2024-04-25T23:52:42.385292+0000","last_fullsized":"2024-04-25T23:52:42.385292+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_clean_scrub_stamp":"2024-04-25T23:48:57.875228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:06:28.527623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"21'4732","reported_seq":4749,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:56.907274+0000","last_change":"2024-04-25T23:49:04.985258+0000","last_active":"2024-04-25T23:52:56.907274+0000","last_peered":"2024-04-25T23:52:56.907274+0000","last_clean":"2024-04-25T23:52:56.907274+0000","last_became_active":"2024-04-25T23:49:04.984516+0000","last_became_peered":"2024-04-25T23:49:04.984516+0000","last_unstale":"2024-04-25T23:52:56.907274+0000","last_undegraded":"2024-04-25T23:52:56.907274+0000","last_fullsized":"2024-04-25T23:52:56.907274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":4732,"log_dups_size":0,"ondisk_log_size":4732,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:43.611787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":89522176,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":1,"num_read_kb":64,"num_write":4706,"num_write_kb":131713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.304971+0000","last_change":"2024-04-25T23:48:58.906152+0000","last_active":"2024-04-25T23:52:42.304971+0000","last_peered":"2024-04-25T23:52:42.304971+0000","last_clean":"2024-04-25T23:52:42.304971+0000","last_became_active":"2024-04-25T23:48:58.905805+0000","last_became_peered":"2024-04-25T23:48:58.905805+0000","last_unstale":"2024-04-25T23:52:42.304971+0000","last_undegraded":"2024-04-25T23:52:42.304971+0000","last_fullsized":"2024-04-25T23:52:42.304971+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_clean_scrub_stamp":"2024-04-25T23:48:57.875228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:05:52.310323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"21'3162","reported_seq":3191,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.801171+0000","last_change":"2024-04-25T23:49:04.978319+0000","last_active":"2024-04-25T23:52:42.801171+0000","last_peered":"2024-04-25T23:52:42.801171+0000","last_clean":"2024-04-25T23:52:42.801171+0000","last_became_active":"2024-04-25T23:49:04.977360+0000","last_became_peered":"2024-04-25T23:49:04.977360+0000","last_unstale":"2024-04-25T23:52:42.801171+0000","last_undegraded":"2024-04-25T23:52:42.801171+0000","last_fullsized":"2024-04-25T23:52:42.801171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":3162,"log_dups_size":0,"ondisk_log_size":3162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:51:25.735100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96731136,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":55,"num_read_kb":103,"num_write":3148,"num_write_kb":156614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.300563+0000","last_change":"2024-04-25T23:48:59.122975+0000","last_active":"2024-04-25T23:52:42.300563+0000","last_peered":"2024-04-25T23:52:42.300563+0000","last_clean":"2024-04-25T23:52:42.300563+0000","last_became_active":"2024-04-25T23:48:59.122193+0000","last_became_peered":"2024-04-25T23:48:59.122193+0000","last_unstale":"2024-04-25T23:52:42.300563+0000","last_undegraded":"2024-04-25T23:52:42.300563+0000","last_fullsized":"2024-04-25T23:52:42.300563+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_clean_scrub_stamp":"2024-04-25T23:48:57.875228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:09:25.578965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"21'7247","reported_seq":7263,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:53:47.589729+0000","last_change":"2024-04-25T23:49:05.092246+0000","last_active":"2024-04-25T23:53:47.589729+0000","last_peered":"2024-04-25T23:53:47.589729+0000","last_clean":"2024-04-25T23:53:47.589729+0000","last_became_active":"2024-04-25T23:49:05.091503+0000","last_became_peered":"2024-04-25T23:49:05.091503+0000","last_unstale":"2024-04-25T23:53:47.589729+0000","last_undegraded":"2024-04-25T23:53:47.589729+0000","last_fullsized":"2024-04-25T23:53:47.589729+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":7247,"log_dups_size":0,"ondisk_log_size":7247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:41:54.323489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":140460032,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7239,"num_write_kb":212436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.304716+0000","last_change":"2024-04-25T23:48:59.120607+0000","last_active":"2024-04-25T23:52:42.304716+0000","last_peered":"2024-04-25T23:52:42.304716+0000","last_clean":"2024-04-25T23:52:42.304716+0000","last_became_active":"2024-04-25T23:48:59.119535+0000","last_became_peered":"2024-04-25T23:48:59.119535+0000","last_unstale":"2024-04-25T23:52:42.304716+0000","last_undegraded":"2024-04-25T23:52:42.304716+0000","last_fullsized":"2024-04-25T23:52:42.304716+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_clean_scrub_stamp":"2024-04-25T23:48:57.875228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:40:05.201442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"20'4309","reported_seq":4437,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.393106+0000","last_change":"2024-04-25T23:49:04.990547+0000","last_active":"2024-04-25T23:52:42.393106+0000","last_peered":"2024-04-25T23:52:42.393106+0000","last_clean":"2024-04-25T23:52:42.393106+0000","last_became_active":"2024-04-25T23:49:04.988031+0000","last_became_peered":"2024-04-25T23:49:04.988031+0000","last_unstale":"2024-04-25T23:52:42.393106+0000","last_undegraded":"2024-04-25T23:52:42.393106+0000","last_fullsized":"2024-04-25T23:52:42.393106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":4309,"log_dups_size":0,"ondisk_log_size":4309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:24:35.524086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100794896,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":148,"num_read_kb":139,"num_write":4335,"num_write_kb":170205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"20'3471","reported_seq":3487,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.320999+0000","last_change":"2024-04-25T23:49:04.995825+0000","last_active":"2024-04-25T23:52:42.320999+0000","last_peered":"2024-04-25T23:52:42.320999+0000","last_clean":"2024-04-25T23:52:42.320999+0000","last_became_active":"2024-04-25T23:49:04.995190+0000","last_became_peered":"2024-04-25T23:49:04.995190+0000","last_unstale":"2024-04-25T23:52:42.320999+0000","last_undegraded":"2024-04-25T23:52:42.320999+0000","last_fullsized":"2024-04-25T23:52:42.320999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":3471,"log_dups_size":0,"ondisk_log_size":3471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:23:45.839121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113246208,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3469,"num_write_kb":172802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.391698+0000","last_change":"2024-04-25T23:48:58.920072+0000","last_active":"2024-04-25T23:52:42.391698+0000","last_peered":"2024-04-25T23:52:42.391698+0000","last_clean":"2024-04-25T23:52:42.391698+0000","last_became_active":"2024-04-25T23:48:58.919554+0000","last_became_peered":"2024-04-25T23:48:58.919554+0000","last_unstale":"2024-04-25T23:52:42.391698+0000","last_undegraded":"2024-04-25T23:52:42.391698+0000","last_fullsized":"2024-04-25T23:52:42.391698+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_clean_scrub_stamp":"2024-04-25T23:48:57.875228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:47:42.230724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"21'3178","reported_seq":3208,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.801745+0000","last_change":"2024-04-25T23:49:05.092897+0000","last_active":"2024-04-25T23:52:42.801745+0000","last_peered":"2024-04-25T23:52:42.801745+0000","last_clean":"2024-04-25T23:52:42.801745+0000","last_became_active":"2024-04-25T23:49:05.092081+0000","last_became_peered":"2024-04-25T23:49:05.092081+0000","last_unstale":"2024-04-25T23:52:42.801745+0000","last_undegraded":"2024-04-25T23:52:42.801745+0000","last_fullsized":"2024-04-25T23:52:42.801745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":3178,"log_dups_size":0,"ondisk_log_size":3178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:15:07.022713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104857600,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":35,"num_read_kb":153,"num_write":3168,"num_write_kb":148628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.326922+0000","last_change":"2024-04-25T23:48:58.910610+0000","last_active":"2024-04-25T23:52:42.326922+0000","last_peered":"2024-04-25T23:52:42.326922+0000","last_clean":"2024-04-25T23:52:42.326922+0000","last_became_active":"2024-04-25T23:48:58.910228+0000","last_became_peered":"2024-04-25T23:48:58.910228+0000","last_unstale":"2024-04-25T23:52:42.326922+0000","last_undegraded":"2024-04-25T23:52:42.326922+0000","last_fullsized":"2024-04-25T23:52:42.326922+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_clean_scrub_stamp":"2024-04-25T23:48:57.875228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:40:43.690806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"21'2866","reported_seq":2883,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.802732+0000","last_change":"2024-04-25T23:49:04.996197+0000","last_active":"2024-04-25T23:52:42.802732+0000","last_peered":"2024-04-25T23:52:42.802732+0000","last_clean":"2024-04-25T23:52:42.802732+0000","last_became_active":"2024-04-25T23:49:04.995711+0000","last_became_peered":"2024-04-25T23:49:04.995711+0000","last_unstale":"2024-04-25T23:52:42.802732+0000","last_undegraded":"2024-04-25T23:52:42.802732+0000","last_fullsized":"2024-04-25T23:52:42.802732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":2866,"log_dups_size":0,"ondisk_log_size":2866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:44:41.716855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":95095128,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":2846,"num_write_kb":142623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.299915+0000","last_change":"2024-04-25T23:48:58.913466+0000","last_active":"2024-04-25T23:52:42.299915+0000","last_peered":"2024-04-25T23:52:42.299915+0000","last_clean":"2024-04-25T23:52:42.299915+0000","last_became_active":"2024-04-25T23:48:58.912966+0000","last_became_peered":"2024-04-25T23:48:58.912966+0000","last_unstale":"2024-04-25T23:52:42.299915+0000","last_undegraded":"2024-04-25T23:52:42.299915+0000","last_fullsized":"2024-04-25T23:52:42.299915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_clean_scrub_stamp":"2024-04-25T23:48:57.875228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:25:56.738346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"21'4357","reported_seq":4378,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:57.666350+0000","last_change":"2024-04-25T23:49:04.977221+0000","last_active":"2024-04-25T23:52:57.666350+0000","last_peered":"2024-04-25T23:52:57.666350+0000","last_clean":"2024-04-25T23:52:57.666350+0000","last_became_active":"2024-04-25T23:49:04.976471+0000","last_became_peered":"2024-04-25T23:49:04.976471+0000","last_unstale":"2024-04-25T23:52:57.666350+0000","last_undegraded":"2024-04-25T23:52:57.666350+0000","last_fullsized":"2024-04-25T23:52:57.666350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":4357,"log_dups_size":0,"ondisk_log_size":4357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:16:51.256351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88211456,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":22,"num_read_kb":16,"num_write":4360,"num_write_kb":167041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.391498+0000","last_change":"2024-04-25T23:48:58.915463+0000","last_active":"2024-04-25T23:52:42.391498+0000","last_peered":"2024-04-25T23:52:42.391498+0000","last_clean":"2024-04-25T23:52:42.391498+0000","last_became_active":"2024-04-25T23:48:58.914379+0000","last_became_peered":"2024-04-25T23:48:58.914379+0000","last_unstale":"2024-04-25T23:52:42.391498+0000","last_undegraded":"2024-04-25T23:52:42.391498+0000","last_fullsized":"2024-04-25T23:52:42.391498+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_clean_scrub_stamp":"2024-04-25T23:48:57.875228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:11:09.852719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":172,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.300390+0000","last_change":"2024-04-25T23:48:59.124853+0000","last_active":"2024-04-25T23:52:42.300390+0000","last_peered":"2024-04-25T23:52:42.300390+0000","last_clean":"2024-04-25T23:52:42.300390+0000","last_became_active":"2024-04-25T23:48:59.122379+0000","last_became_peered":"2024-04-25T23:48:59.122379+0000","last_unstale":"2024-04-25T23:52:42.300390+0000","last_undegraded":"2024-04-25T23:52:42.300390+0000","last_fullsized":"2024-04-25T23:52:42.300390+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_clean_scrub_stamp":"2024-04-25T23:48:57.875228+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:57:31.186906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"21'3897","reported_seq":3938,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.802677+0000","last_change":"2024-04-25T23:49:04.980999+0000","last_active":"2024-04-25T23:52:42.802677+0000","last_peered":"2024-04-25T23:52:42.802677+0000","last_clean":"2024-04-25T23:52:42.802677+0000","last_became_active":"2024-04-25T23:49:04.980272+0000","last_became_peered":"2024-04-25T23:49:04.980272+0000","last_unstale":"2024-04-25T23:52:42.802677+0000","last_undegraded":"2024-04-25T23:52:42.802677+0000","last_fullsized":"2024-04-25T23:52:42.802677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":3897,"log_dups_size":0,"ondisk_log_size":3897,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:43:10.867606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100794368,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":94,"num_read_kb":77,"num_write":3859,"num_write_kb":201602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"21'7829","reported_seq":7966,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:53:57.613005+0000","last_change":"2024-04-25T23:49:05.094703+0000","last_active":"2024-04-25T23:53:57.613005+0000","last_peered":"2024-04-25T23:53:57.613005+0000","last_clean":"2024-04-25T23:53:57.613005+0000","last_became_active":"2024-04-25T23:49:05.094007+0000","last_became_peered":"2024-04-25T23:49:05.094007+0000","last_unstale":"2024-04-25T23:53:57.613005+0000","last_undegraded":"2024-04-25T23:53:57.613005+0000","last_fullsized":"2024-04-25T23:53:57.613005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":7829,"log_dups_size":0,"ondisk_log_size":7829,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:33:21.090339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":95027712,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":414,"num_read_kb":363,"num_write":5838,"num_write_kb":183309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"21'4056","reported_seq":4079,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:53:47.643827+0000","last_change":"2024-04-25T23:49:04.983367+0000","last_active":"2024-04-25T23:53:47.643827+0000","last_peered":"2024-04-25T23:53:47.643827+0000","last_clean":"2024-04-25T23:53:47.643827+0000","last_became_active":"2024-04-25T23:49:04.982660+0000","last_became_peered":"2024-04-25T23:49:04.982660+0000","last_unstale":"2024-04-25T23:53:47.643827+0000","last_undegraded":"2024-04-25T23:53:47.643827+0000","last_fullsized":"2024-04-25T23:53:47.643827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":4056,"log_dups_size":0,"ondisk_log_size":4056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:34:08.771639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123396624,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":13,"num_read_kb":11,"num_write":4049,"num_write_kb":164094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"20'3795","reported_seq":3811,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.302007+0000","last_change":"2024-04-25T23:49:04.991004+0000","last_active":"2024-04-25T23:52:42.302007+0000","last_peered":"2024-04-25T23:52:42.302007+0000","last_clean":"2024-04-25T23:52:42.302007+0000","last_became_active":"2024-04-25T23:49:04.990144+0000","last_became_peered":"2024-04-25T23:49:04.990144+0000","last_unstale":"2024-04-25T23:52:42.302007+0000","last_undegraded":"2024-04-25T23:52:42.302007+0000","last_fullsized":"2024-04-25T23:52:42.302007+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":3795,"log_dups_size":0,"ondisk_log_size":3795,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:18:20.437365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113246208,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3792,"num_write_kb":169093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"21'3586","reported_seq":3613,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.794253+0000","last_change":"2024-04-25T23:49:04.988518+0000","last_active":"2024-04-25T23:52:42.794253+0000","last_peered":"2024-04-25T23:52:42.794253+0000","last_clean":"2024-04-25T23:52:42.794253+0000","last_became_active":"2024-04-25T23:49:04.987663+0000","last_became_peered":"2024-04-25T23:49:04.987663+0000","last_unstale":"2024-04-25T23:52:42.794253+0000","last_undegraded":"2024-04-25T23:52:42.794253+0000","last_fullsized":"2024-04-25T23:52:42.794253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":3586,"log_dups_size":0,"ondisk_log_size":3586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:08:28.379346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112721920,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":45,"num_read_kb":34,"num_write":3560,"num_write_kb":152196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"21'4479","reported_seq":4500,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.800823+0000","last_change":"2024-04-25T23:49:04.997861+0000","last_active":"2024-04-25T23:52:42.800823+0000","last_peered":"2024-04-25T23:52:42.800823+0000","last_clean":"2024-04-25T23:52:42.800823+0000","last_became_active":"2024-04-25T23:49:04.996781+0000","last_became_peered":"2024-04-25T23:49:04.996781+0000","last_unstale":"2024-04-25T23:52:42.800823+0000","last_undegraded":"2024-04-25T23:52:42.800823+0000","last_fullsized":"2024-04-25T23:52:42.800823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":4479,"log_dups_size":0,"ondisk_log_size":4479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:48:47.865840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":127338152,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":8,"num_read_kb":7,"num_write":4466,"num_write_kb":191484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"21'5770","reported_seq":5861,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.888218+0000","last_change":"2024-04-25T23:49:04.998785+0000","last_active":"2024-04-25T23:52:42.888218+0000","last_peered":"2024-04-25T23:52:42.888218+0000","last_clean":"2024-04-25T23:52:42.888218+0000","last_became_active":"2024-04-25T23:49:04.997977+0000","last_became_peered":"2024-04-25T23:49:04.997977+0000","last_unstale":"2024-04-25T23:52:42.888218+0000","last_undegraded":"2024-04-25T23:52:42.888218+0000","last_fullsized":"2024-04-25T23:52:42.888218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":5770,"log_dups_size":0,"ondisk_log_size":5770,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:13:53.271710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":116271984,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":286,"num_read_kb":214,"num_write":6036,"num_write_kb":177360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"21'3327","reported_seq":3343,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.804453+0000","last_change":"2024-04-25T23:49:04.998698+0000","last_active":"2024-04-25T23:52:42.804453+0000","last_peered":"2024-04-25T23:52:42.804453+0000","last_clean":"2024-04-25T23:52:42.804453+0000","last_became_active":"2024-04-25T23:49:04.997986+0000","last_became_peered":"2024-04-25T23:49:04.997986+0000","last_unstale":"2024-04-25T23:52:42.804453+0000","last_undegraded":"2024-04-25T23:52:42.804453+0000","last_fullsized":"2024-04-25T23:52:42.804453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":3327,"log_dups_size":0,"ondisk_log_size":3327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:59:11.386821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663296,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3277,"num_write_kb":152452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"21'3069","reported_seq":3085,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.796063+0000","last_change":"2024-04-25T23:49:04.990446+0000","last_active":"2024-04-25T23:52:42.796063+0000","last_peered":"2024-04-25T23:52:42.796063+0000","last_clean":"2024-04-25T23:52:42.796063+0000","last_became_active":"2024-04-25T23:49:04.989709+0000","last_became_peered":"2024-04-25T23:49:04.989709+0000","last_unstale":"2024-04-25T23:52:42.796063+0000","last_undegraded":"2024-04-25T23:52:42.796063+0000","last_fullsized":"2024-04-25T23:52:42.796063+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":3069,"log_dups_size":0,"ondisk_log_size":3069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:22:01.360709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3021,"num_write_kb":144132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"20'4577","reported_seq":4601,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.299824+0000","last_change":"2024-04-25T23:49:04.998536+0000","last_active":"2024-04-25T23:52:42.299824+0000","last_peered":"2024-04-25T23:52:42.299824+0000","last_clean":"2024-04-25T23:52:42.299824+0000","last_became_active":"2024-04-25T23:49:04.997718+0000","last_became_peered":"2024-04-25T23:49:04.997718+0000","last_unstale":"2024-04-25T23:52:42.299824+0000","last_undegraded":"2024-04-25T23:52:42.299824+0000","last_fullsized":"2024-04-25T23:52:42.299824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":4577,"log_dups_size":0,"ondisk_log_size":4577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:42:48.145557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117440512,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":4584,"num_write_kb":172941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"21'4926","reported_seq":4942,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.806464+0000","last_change":"2024-04-25T23:49:04.988912+0000","last_active":"2024-04-25T23:52:42.806464+0000","last_peered":"2024-04-25T23:52:42.806464+0000","last_clean":"2024-04-25T23:52:42.806464+0000","last_became_active":"2024-04-25T23:49:04.987367+0000","last_became_peered":"2024-04-25T23:49:04.987367+0000","last_unstale":"2024-04-25T23:52:42.806464+0000","last_undegraded":"2024-04-25T23:52:42.806464+0000","last_fullsized":"2024-04-25T23:52:42.806464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":4926,"log_dups_size":0,"ondisk_log_size":4926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:11:40.900415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":149553152,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4881,"num_write_kb":166152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"21'7343","reported_seq":7369,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:55.941996+0000","last_change":"2024-04-25T23:49:04.997739+0000","last_active":"2024-04-25T23:52:55.941996+0000","last_peered":"2024-04-25T23:52:55.941996+0000","last_clean":"2024-04-25T23:52:55.941996+0000","last_became_active":"2024-04-25T23:49:04.996841+0000","last_became_peered":"2024-04-25T23:49:04.996841+0000","last_unstale":"2024-04-25T23:52:55.941996+0000","last_undegraded":"2024-04-25T23:52:55.941996+0000","last_fullsized":"2024-04-25T23:52:55.941996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":7343,"log_dups_size":0,"ondisk_log_size":7343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:08:44.872965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83346448,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":33,"num_read_kb":88,"num_write":7349,"num_write_kb":122645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":3560846088,"num_objects":877,"num_object_clones":0,"num_object_copies":1754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":877,"num_whiteouts":0,"num_read":2507,"num_read_kb":2518,"num_write":155692,"num_write_kb":5620473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":6782795776,"data_stored":6820341912,"data_compressed":194288578,"data_compressed_allocated":202072064,"data_compressed_original":243892224,"omap_allocated":0,"internal_metadata":0},"log_size":157484,"ondisk_log_size":157484,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":12118,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542206,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":388020,"kb_used_data":298604,"kb_used_omap":1,"kb_used_meta":89406,"kb_avail":93983820,"statfs":{"total":96636764160,"available":96239431680,"internally_reserved":0,"allocated":305770496,"data_stored":306544866,"data_compressed":6553827,"data_compressed_allocated":6823936,"data_compressed_original":7905280,"omap_allocated":1593,"internal_metadata":91552199},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542208,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1212816,"kb_used_data":929032,"kb_used_omap":1,"kb_used_meta":283774,"kb_avail":93159024,"statfs":{"total":96636764160,"available":95394840576,"internally_reserved":0,"allocated":951328768,"data_stored":954362994,"data_compressed":23747576,"data_compressed_allocated":24735744,"data_compressed_original":28565504,"omap_allocated":1595,"internal_metadata":290585029},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542206,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1859000,"kb_used_data":1332336,"kb_used_omap":1,"kb_used_meta":526654,"kb_avail":92512840,"statfs":{"total":96636764160,"available":94733148160,"internally_reserved":0,"allocated":1364312064,"data_stored":1378232810,"data_compressed":53769970,"data_compressed_allocated":55988224,"data_compressed_original":70811648,"omap_allocated":1595,"internal_metadata":539294149},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574910,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1460108,"kb_used_data":1031236,"kb_used_omap":1,"kb_used_meta":428862,"kb_avail":92911732,"statfs":{"total":96636764160,"available":95141613568,"internally_reserved":0,"allocated":1055985664,"data_stored":1063157334,"data_compressed":33533197,"data_compressed_allocated":34906112,"data_compressed_original":43012096,"omap_allocated":1590,"internal_metadata":439155146},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574911,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1418012,"kb_used_data":1062036,"kb_used_omap":1,"kb_used_meta":355966,"kb_avail":92953828,"statfs":{"total":96636764160,"available":95184719872,"internally_reserved":0,"allocated":1087524864,"data_stored":1089954434,"data_compressed":21531874,"data_compressed_allocated":22360064,"data_compressed_original":25559040,"omap_allocated":1593,"internal_metadata":364509639},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574910,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":424748,"kb_used_data":303332,"kb_used_omap":1,"kb_used_meta":121406,"kb_avail":93947092,"statfs":{"total":96636764160,"available":96201822208,"internally_reserved":0,"allocated":310611968,"data_stored":314944206,"data_compressed":11229575,"data_compressed_allocated":11784192,"data_compressed_original":16420864,"omap_allocated":1595,"internal_metadata":124320197},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":861340,"kb_used_data":623380,"kb_used_omap":1,"kb_used_meta":237950,"kb_avail":93510500,"statfs":{"total":96636764160,"available":95754752000,"internally_reserved":0,"allocated":638341120,"data_stored":641442682,"data_compressed":20262986,"data_compressed_allocated":21012480,"data_compressed_original":24612864,"omap_allocated":1595,"internal_metadata":243661253},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574910,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1391756,"kb_used_data":1045508,"kb_used_omap":1,"kb_used_meta":346238,"kb_avail":92980084,"statfs":{"total":96636764160,"available":95211606016,"internally_reserved":0,"allocated":1070600192,"data_stored":1073278922,"data_compressed":23671691,"data_compressed_allocated":24526848,"data_compressed_original":28028928,"omap_allocated":1595,"internal_metadata":354548165},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":6059,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":6059,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1070350336,"data_stored":1072685184,"data_compressed":23665632,"data_compressed_allocated":24494080,"data_compressed_original":27516928,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":638144512,"data_stored":641377872,"data_compressed":20262986,"data_compressed_allocated":21012480,"data_compressed_original":24612864,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":310415360,"data_stored":314879396,"data_compressed":11229575,"data_compressed_allocated":11784192,"data_compressed_original":16420864,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1087328256,"data_stored":1089889624,"data_compressed":21531874,"data_compressed_allocated":22360064,"data_compressed_original":25559040,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1364115456,"data_stored":1378168000,"data_compressed":53769970,"data_compressed_allocated":55988224,"data_compressed_original":70811648,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":951132160,"data_stored":954298184,"data_compressed":23747576,"data_compressed_allocated":24735744,"data_compressed_original":28565504,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":305573888,"data_stored":306480056,"data_compressed":6553827,"data_compressed_allocated":6823936,"data_compressed_original":7905280,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1055735808,"data_stored":1062563596,"data_compressed":33527138,"data_compressed_allocated":34873344,"data_compressed_original":42500096,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T23:53:59.190 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T23:53:59.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.328+0000 7feb28bcf640 1 Processor -- start 2024-04-25T23:53:59.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.328+0000 7feb28bcf640 1 -- start start 2024-04-25T23:53:59.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.328+0000 7feb28bcf640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7feb24142740 0x7feb24070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:53:59.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.328+0000 7feb28bcf640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7feb2406ea90 0x7feb24146670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:53:59.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.328+0000 7feb28bcf640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb24072130 con 0x7feb24071d50 2024-04-25T23:53:59.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.328+0000 7feb28bcf640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7feb240adb10 con 0x7feb2406ea90 2024-04-25T23:53:59.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.328+0000 7feb28bcf640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7feb241461e0 con 0x7feb24142740 2024-04-25T23:53:59.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.332+0000 7feb21d9b640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7feb24142740 0x7feb24070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:53:59.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.332+0000 7feb21d9b640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7feb24142740 0x7feb24070ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3301/0 says I am v2:172.21.15.57:48592/0 (socket says 172.21.15.57:48592) 2024-04-25T23:53:59.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.332+0000 7feb21d9b640 1 -- 172.21.15.57:0/3342662615 learned_addr learned my addr 172.21.15.57:0/3342662615 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:53:59.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.332+0000 7feb21d9b640 1 -- 172.21.15.57:0/3342662615 >> v2:172.21.15.80:3300/0 conn(0x7feb2406ea90 msgr2=0x7feb24146670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:53:59.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.332+0000 7feb21d9b640 1 --2- 172.21.15.57:0/3342662615 >> v2:172.21.15.80:3300/0 conn(0x7feb2406ea90 0x7feb24146670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:53:59.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.332+0000 7feb21d9b640 1 -- 172.21.15.57:0/3342662615 >> v1:172.21.15.80:6789/0 conn(0x7feb24071d50 legacy=0x7feb24142030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:53:59.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.332+0000 7feb21d9b640 1 -- 172.21.15.57:0/3342662615 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb24071640 con 0x7feb24142740 2024-04-25T23:53:59.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.332+0000 7feb21d9b640 1 --2- 172.21.15.57:0/3342662615 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7feb24142740 0x7feb24070ff0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7feb18009870 tx=0x7feb1801c320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=461802d68c65e400 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:53:59.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.332+0000 7feb2159a640 1 -- 172.21.15.57:0/3342662615 <== mon.2 v2:172.21.15.57:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb180b0070 con 0x7feb24142740 2024-04-25T23:53:59.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.332+0000 7feb2159a640 1 -- 172.21.15.57:0/3342662615 <== mon.2 v2:172.21.15.57:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb180ab600 con 0x7feb24142740 2024-04-25T23:53:59.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.332+0000 7feb2159a640 1 -- 172.21.15.57:0/3342662615 <== mon.2 v2:172.21.15.57:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb180ab9d0 con 0x7feb24142740 2024-04-25T23:53:59.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.332+0000 7feb28bcf640 1 -- 172.21.15.57:0/3342662615 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7feb24142740 msgr2=0x7feb24070ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:53:59.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.332+0000 7feb28bcf640 1 --2- 172.21.15.57:0/3342662615 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7feb24142740 0x7feb24070ff0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7feb18009870 tx=0x7feb1801c320 comp rx=0 tx=0).stop 2024-04-25T23:53:59.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.332+0000 7feb28bcf640 1 -- 172.21.15.57:0/3342662615 shutdown_connections 2024-04-25T23:53:59.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.332+0000 7feb28bcf640 1 --2- 172.21.15.57:0/3342662615 >> v2:172.21.15.80:3300/0 conn(0x7feb2406ea90 0x7feb24146670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:53:59.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.332+0000 7feb28bcf640 1 --2- 172.21.15.57:0/3342662615 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7feb24142740 0x7feb24070ff0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:53:59.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.332+0000 7feb28bcf640 1 -- 172.21.15.57:0/3342662615 >> 172.21.15.57:0/3342662615 conn(0x7feb240b9110 msgr2=0x7feb240b9520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:53:59.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.332+0000 7feb28bcf640 1 -- 172.21.15.57:0/3342662615 shutdown_connections 2024-04-25T23:53:59.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.332+0000 7feb28bcf640 1 -- 172.21.15.57:0/3342662615 wait complete. 2024-04-25T23:53:59.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.336+0000 7feb28bcf640 1 Processor -- start 2024-04-25T23:53:59.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.336+0000 7feb28bcf640 1 -- start start 2024-04-25T23:53:59.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.336+0000 7feb28bcf640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb2406ea90 0x7feb24138f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:53:59.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.336+0000 7feb28bcf640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7feb24071d50 0x7feb2413d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:53:59.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.336+0000 7feb28bcf640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feb24142740 0x7feb241dec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:53:59.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.336+0000 7feb28bcf640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7feb241501d0 con 0x7feb2406ea90 2024-04-25T23:53:59.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.336+0000 7feb28bcf640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7feb24150050 con 0x7feb24142740 2024-04-25T23:53:59.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.336+0000 7feb28bcf640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7feb24150350 con 0x7feb24071d50 2024-04-25T23:53:59.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.336+0000 7feb21d9b640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7feb24071d50 0x7feb2413d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:53:59.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.336+0000 7feb2259c640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb2406ea90 0x7feb24138f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:53:59.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.336+0000 7feb21d9b640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7feb24071d50 0x7feb2413d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3301/0 says I am v2:172.21.15.57:48600/0 (socket says 172.21.15.57:48600) 2024-04-25T23:53:59.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.336+0000 7feb21d9b640 1 -- 172.21.15.57:0/4143679954 learned_addr learned my addr 172.21.15.57:0/4143679954 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:53:59.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.336+0000 7feb21d9b640 1 -- 172.21.15.57:0/4143679954 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feb24142740 msgr2=0x7feb241dec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:53:59.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.336+0000 7feb22d9d640 1 --2- 172.21.15.57:0/4143679954 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feb24142740 0x7feb241dec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:53:59.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.336+0000 7feb21d9b640 1 --2- 172.21.15.57:0/4143679954 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feb24142740 0x7feb241dec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:53:59.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.336+0000 7feb21d9b640 1 -- 172.21.15.57:0/4143679954 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb2406ea90 msgr2=0x7feb24138f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:53:59.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.336+0000 7feb21d9b640 1 --2- 172.21.15.57:0/4143679954 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb2406ea90 0x7feb24138f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:53:59.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.336+0000 7feb21d9b640 1 -- 172.21.15.57:0/4143679954 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb2413c8a0 con 0x7feb24071d50 2024-04-25T23:53:59.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.336+0000 7feb2259c640 1 --2- 172.21.15.57:0/4143679954 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb2406ea90 0x7feb24138f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T23:53:59.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.336+0000 7feb21d9b640 1 --2- 172.21.15.57:0/4143679954 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7feb24071d50 0x7feb2413d670 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7feb1801cbc0 tx=0x7feb1801ab60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:53:59.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.336+0000 7feb12ffd640 1 -- 172.21.15.57:0/4143679954 <== mon.2 v2:172.21.15.57:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb180ba020 con 0x7feb24071d50 2024-04-25T23:53:59.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.336+0000 7feb12ffd640 1 -- 172.21.15.57:0/4143679954 <== mon.2 v2:172.21.15.57:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb180b6620 con 0x7feb24071d50 2024-04-25T23:53:59.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.336+0000 7feb28bcf640 1 -- 172.21.15.57:0/4143679954 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb24137810 con 0x7feb24071d50 2024-04-25T23:53:59.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.336+0000 7feb12ffd640 1 -- 172.21.15.57:0/4143679954 <== mon.2 v2:172.21.15.57:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb180b0040 con 0x7feb24071d50 2024-04-25T23:53:59.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.336+0000 7feb28bcf640 1 -- 172.21.15.57:0/4143679954 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7feb2413dd80 con 0x7feb24071d50 2024-04-25T23:53:59.327 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.336+0000 7feb28bcf640 1 -- 172.21.15.57:0/4143679954 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb240761e0 con 0x7feb24071d50 2024-04-25T23:53:59.327 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.336+0000 7feb12ffd640 1 -- 172.21.15.57:0/4143679954 <== mon.2 v2:172.21.15.57:3301/0 4 ==== mgrmap(e 5) ==== 87410+0+0 (secure 0 0 0) 0x7feb180bd020 con 0x7feb24071d50 2024-04-25T23:53:59.327 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.340+0000 7feb12ffd640 1 --2- 172.21.15.57:0/4143679954 >> v2:172.21.15.80:6816/2493457677 conn(0x7feb04067880 0x7feb04069d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:53:59.327 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.340+0000 7feb12ffd640 1 -- 172.21.15.57:0/4143679954 <== mon.2 v2:172.21.15.57:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 4525+0+0 (secure 0 0 0) 0x7feb1811f9e0 con 0x7feb24071d50 2024-04-25T23:53:59.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.340+0000 7feb2259c640 1 --2- 172.21.15.57:0/4143679954 >> v2:172.21.15.80:6816/2493457677 conn(0x7feb04067880 0x7feb04069d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:53:59.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.340+0000 7feb2259c640 1 --2- 172.21.15.57:0/4143679954 >> v2:172.21.15.80:6816/2493457677 conn(0x7feb04067880 0x7feb04069d40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7feb14007fd0 tx=0x7feb14009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:53:59.328 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.344+0000 7feb12ffd640 1 -- 172.21.15.57:0/4143679954 <== mon.2 v2:172.21.15.57:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7feb180c4030 con 0x7feb24071d50 2024-04-25T23:53:59.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.500+0000 7feb28bcf640 1 -- 172.21.15.57:0/4143679954 --> v2:172.21.15.80:6816/2493457677 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7feb2414d2c0 con 0x7feb04067880 2024-04-25T23:53:59.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.512+0000 7feb12ffd640 1 -- 172.21.15.57:0/4143679954 <== mgr.4100 v2:172.21.15.80:6816/2493457677 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+117689 (secure 0 0 0) 0x7feb2414d2c0 con 0x7feb04067880 2024-04-25T23:53:59.497 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-25T23:53:59.498 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2024-04-25T23:53:59.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.512+0000 7feb28bcf640 1 -- 172.21.15.57:0/4143679954 >> v2:172.21.15.80:6816/2493457677 conn(0x7feb04067880 msgr2=0x7feb04069d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:53:59.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.512+0000 7feb28bcf640 1 --2- 172.21.15.57:0/4143679954 >> v2:172.21.15.80:6816/2493457677 conn(0x7feb04067880 0x7feb04069d40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7feb14007fd0 tx=0x7feb14009040 comp rx=0 tx=0).stop 2024-04-25T23:53:59.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.512+0000 7feb28bcf640 1 -- 172.21.15.57:0/4143679954 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7feb24071d50 msgr2=0x7feb2413d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:53:59.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.512+0000 7feb28bcf640 1 --2- 172.21.15.57:0/4143679954 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7feb24071d50 0x7feb2413d670 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7feb1801cbc0 tx=0x7feb1801ab60 comp rx=0 tx=0).stop 2024-04-25T23:53:59.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.512+0000 7feb28bcf640 1 -- 172.21.15.57:0/4143679954 shutdown_connections 2024-04-25T23:53:59.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.512+0000 7feb28bcf640 1 --2- 172.21.15.57:0/4143679954 >> v2:172.21.15.80:6816/2493457677 conn(0x7feb04067880 0x7feb04069d40 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:53:59.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.512+0000 7feb28bcf640 1 --2- 172.21.15.57:0/4143679954 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feb24142740 0x7feb241dec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:53:59.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.512+0000 7feb28bcf640 1 --2- 172.21.15.57:0/4143679954 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7feb24071d50 0x7feb2413d670 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:53:59.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.512+0000 7feb28bcf640 1 --2- 172.21.15.57:0/4143679954 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7feb2406ea90 0x7feb24138f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:53:59.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.516+0000 7feb28bcf640 1 -- 172.21.15.57:0/4143679954 >> 172.21.15.57:0/4143679954 conn(0x7feb240b9110 msgr2=0x7feb24070a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:53:59.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.516+0000 7feb28bcf640 1 -- 172.21.15.57:0/4143679954 shutdown_connections 2024-04-25T23:53:59.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.516+0000 7feb28bcf640 1 -- 172.21.15.57:0/4143679954 wait complete. 2024-04-25T23:53:59.517 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":166,"stamp":"2024-04-25T23:53:58.919008+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3561436456,"num_objects":879,"num_object_clones":0,"num_object_copies":1758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":879,"num_whiteouts":0,"num_read":2623,"num_read_kb":2618,"num_write":155851,"num_write_kb":5622889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157601,"ondisk_log_size":157601,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":9015800,"kb_used_data":6625464,"kb_used_omap":12,"kb_used_meta":2390259,"kb_avail":745958920,"statfs":{"total":773094113280,"available":763861934080,"internally_reserved":0,"allocated":6784475136,"data_stored":6821918248,"data_compressed":194300696,"data_compressed_allocated":202137600,"data_compressed_original":244916224,"omap_allocated":12751,"internal_metadata":2447625777},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":16384,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":5,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003287"},"pg_stats":[{"pgid":"3.1f","version":"20'4490","reported_seq":4529,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.383532+0000","last_change":"2024-04-25T23:49:04.991824+0000","last_active":"2024-04-25T23:52:42.383532+0000","last_peered":"2024-04-25T23:52:42.383532+0000","last_clean":"2024-04-25T23:52:42.383532+0000","last_became_active":"2024-04-25T23:49:04.991059+0000","last_became_peered":"2024-04-25T23:49:04.991059+0000","last_unstale":"2024-04-25T23:52:42.383532+0000","last_undegraded":"2024-04-25T23:52:42.383532+0000","last_fullsized":"2024-04-25T23:52:42.383532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":4490,"log_dups_size":0,"ondisk_log_size":4490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:06:08.003629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121634816,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":93,"num_read_kb":81,"num_write":4513,"num_write_kb":218308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"21'5438","reported_seq":5465,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:50.093018+0000","last_change":"2024-04-25T23:49:04.991993+0000","last_active":"2024-04-25T23:52:50.093018+0000","last_peered":"2024-04-25T23:52:50.093018+0000","last_clean":"2024-04-25T23:52:50.093018+0000","last_became_active":"2024-04-25T23:49:04.991224+0000","last_became_peered":"2024-04-25T23:49:04.991224+0000","last_unstale":"2024-04-25T23:52:50.093018+0000","last_undegraded":"2024-04-25T23:52:50.093018+0000","last_fullsized":"2024-04-25T23:52:50.093018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":5438,"log_dups_size":0,"ondisk_log_size":5438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:49:45.384039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":45,"num_read_kb":37,"num_write":5439,"num_write_kb":172929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"21'6313","reported_seq":6337,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:56.683805+0000","last_change":"2024-04-25T23:49:04.984448+0000","last_active":"2024-04-25T23:52:56.683805+0000","last_peered":"2024-04-25T23:52:56.683805+0000","last_clean":"2024-04-25T23:52:56.683805+0000","last_became_active":"2024-04-25T23:49:04.982204+0000","last_became_peered":"2024-04-25T23:49:04.982204+0000","last_unstale":"2024-04-25T23:52:56.683805+0000","last_undegraded":"2024-04-25T23:52:56.683805+0000","last_fullsized":"2024-04-25T23:52:56.683805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":6313,"log_dups_size":0,"ondisk_log_size":6313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:23:55.891971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":150994944,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":30,"num_read_kb":85,"num_write":6420,"num_write_kb":204243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"20'3855","reported_seq":3874,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.391433+0000","last_change":"2024-04-25T23:49:04.980119+0000","last_active":"2024-04-25T23:52:42.391433+0000","last_peered":"2024-04-25T23:52:42.391433+0000","last_clean":"2024-04-25T23:52:42.391433+0000","last_became_active":"2024-04-25T23:49:04.979323+0000","last_became_peered":"2024-04-25T23:49:04.979323+0000","last_unstale":"2024-04-25T23:52:42.391433+0000","last_undegraded":"2024-04-25T23:52:42.391433+0000","last_fullsized":"2024-04-25T23:52:42.391433+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":3855,"log_dups_size":0,"ondisk_log_size":3855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:09:46.513536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96535048,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":17,"num_read_kb":12,"num_write":3868,"num_write_kb":155287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"21'4570","reported_seq":4585,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.802622+0000","last_change":"2024-04-25T23:49:05.002350+0000","last_active":"2024-04-25T23:52:42.802622+0000","last_peered":"2024-04-25T23:52:42.802622+0000","last_clean":"2024-04-25T23:52:42.802622+0000","last_became_active":"2024-04-25T23:49:05.001246+0000","last_became_peered":"2024-04-25T23:49:05.001246+0000","last_unstale":"2024-04-25T23:52:42.802622+0000","last_undegraded":"2024-04-25T23:52:42.802622+0000","last_fullsized":"2024-04-25T23:52:42.802622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":4570,"log_dups_size":0,"ondisk_log_size":4570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:27:02.668986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":107675648,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4523,"num_write_kb":223556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"21'6959","reported_seq":6974,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:50.115868+0000","last_change":"2024-04-25T23:49:04.984321+0000","last_active":"2024-04-25T23:52:50.115868+0000","last_peered":"2024-04-25T23:52:50.115868+0000","last_clean":"2024-04-25T23:52:50.115868+0000","last_became_active":"2024-04-25T23:49:04.983518+0000","last_became_peered":"2024-04-25T23:49:04.983518+0000","last_unstale":"2024-04-25T23:52:50.115868+0000","last_undegraded":"2024-04-25T23:52:50.115868+0000","last_fullsized":"2024-04-25T23:52:50.115868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":6959,"log_dups_size":0,"ondisk_log_size":6959,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:31:32.872285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":138412032,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6933,"num_write_kb":206597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"21'5764","reported_seq":5780,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:53:01.763608+0000","last_change":"2024-04-25T23:49:04.985251+0000","last_active":"2024-04-25T23:53:01.763608+0000","last_peered":"2024-04-25T23:53:01.763608+0000","last_clean":"2024-04-25T23:53:01.763608+0000","last_became_active":"2024-04-25T23:49:04.984378+0000","last_became_peered":"2024-04-25T23:49:04.984378+0000","last_unstale":"2024-04-25T23:53:01.763608+0000","last_undegraded":"2024-04-25T23:53:01.763608+0000","last_fullsized":"2024-04-25T23:53:01.763608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":5764,"log_dups_size":0,"ondisk_log_size":5764,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:43:34.630095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121634816,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5717,"num_write_kb":246662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"21'7903","reported_seq":7920,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:53:00.638258+0000","last_change":"2024-04-25T23:49:04.988246+0000","last_active":"2024-04-25T23:53:00.638258+0000","last_peered":"2024-04-25T23:53:00.638258+0000","last_clean":"2024-04-25T23:53:00.638258+0000","last_became_active":"2024-04-25T23:49:04.987468+0000","last_became_peered":"2024-04-25T23:49:04.987468+0000","last_unstale":"2024-04-25T23:53:00.638258+0000","last_undegraded":"2024-04-25T23:53:00.638258+0000","last_fullsized":"2024-04-25T23:53:00.638258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":7903,"log_dups_size":0,"ondisk_log_size":7903,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:21:42.911815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":130023936,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":7908,"num_write_kb":222727,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"21'5984","reported_seq":6008,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:53.277327+0000","last_change":"2024-04-25T23:49:04.989029+0000","last_active":"2024-04-25T23:52:53.277327+0000","last_peered":"2024-04-25T23:52:53.277327+0000","last_clean":"2024-04-25T23:52:53.277327+0000","last_became_active":"2024-04-25T23:49:04.987729+0000","last_became_peered":"2024-04-25T23:49:04.987729+0000","last_unstale":"2024-04-25T23:52:53.277327+0000","last_undegraded":"2024-04-25T23:52:53.277327+0000","last_fullsized":"2024-04-25T23:52:53.277327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":5984,"log_dups_size":0,"ondisk_log_size":5984,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:34:36.865614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121634816,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":5995,"num_write_kb":201733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"21'3557","reported_seq":3881,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.801711+0000","last_change":"2024-04-25T23:49:04.986698+0000","last_active":"2024-04-25T23:52:42.801711+0000","last_peered":"2024-04-25T23:52:42.801711+0000","last_clean":"2024-04-25T23:52:42.801711+0000","last_became_active":"2024-04-25T23:49:04.985957+0000","last_became_peered":"2024-04-25T23:49:04.985957+0000","last_unstale":"2024-04-25T23:52:42.801711+0000","last_undegraded":"2024-04-25T23:52:42.801711+0000","last_fullsized":"2024-04-25T23:52:42.801711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":3557,"log_dups_size":0,"ondisk_log_size":3557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:07:54.597105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":77201936,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":1028,"num_read_kb":921,"num_write":3751,"num_write_kb":130317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"21'7645","reported_seq":7661,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:53:00.534834+0000","last_change":"2024-04-25T23:49:04.992705+0000","last_active":"2024-04-25T23:53:00.534834+0000","last_peered":"2024-04-25T23:53:00.534834+0000","last_clean":"2024-04-25T23:53:00.534834+0000","last_became_active":"2024-04-25T23:49:04.990812+0000","last_became_peered":"2024-04-25T23:49:04.990812+0000","last_unstale":"2024-04-25T23:53:00.534834+0000","last_undegraded":"2024-04-25T23:53:00.534834+0000","last_fullsized":"2024-04-25T23:53:00.534834+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":7645,"log_dups_size":0,"ondisk_log_size":7645,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:27:48.304794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":138412032,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7636,"num_write_kb":198278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"21'5030","reported_seq":5062,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:52.943801+0000","last_change":"2024-04-25T23:49:04.990899+0000","last_active":"2024-04-25T23:52:52.943801+0000","last_peered":"2024-04-25T23:52:52.943801+0000","last_clean":"2024-04-25T23:52:52.943801+0000","last_became_active":"2024-04-25T23:49:04.988250+0000","last_became_peered":"2024-04-25T23:49:04.988250+0000","last_unstale":"2024-04-25T23:52:52.943801+0000","last_undegraded":"2024-04-25T23:52:52.943801+0000","last_fullsized":"2024-04-25T23:52:52.943801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":5030,"log_dups_size":0,"ondisk_log_size":5030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:53:15.377369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":95029072,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":68,"num_read_kb":57,"num_write":5006,"num_write_kb":140314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.385292+0000","last_change":"2024-04-25T23:48:58.911464+0000","last_active":"2024-04-25T23:52:42.385292+0000","last_peered":"2024-04-25T23:52:42.385292+0000","last_clean":"2024-04-25T23:52:42.385292+0000","last_became_active":"2024-04-25T23:48:58.911101+0000","last_became_peered":"2024-04-25T23:48:58.911101+0000","last_unstale":"2024-04-25T23:52:42.385292+0000","last_undegraded":"2024-04-25T23:52:42.385292+0000","last_fullsized":"2024-04-25T23:52:42.385292+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_clean_scrub_stamp":"2024-04-25T23:48:57.875228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:06:28.527623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"21'4732","reported_seq":4749,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:56.907274+0000","last_change":"2024-04-25T23:49:04.985258+0000","last_active":"2024-04-25T23:52:56.907274+0000","last_peered":"2024-04-25T23:52:56.907274+0000","last_clean":"2024-04-25T23:52:56.907274+0000","last_became_active":"2024-04-25T23:49:04.984516+0000","last_became_peered":"2024-04-25T23:49:04.984516+0000","last_unstale":"2024-04-25T23:52:56.907274+0000","last_undegraded":"2024-04-25T23:52:56.907274+0000","last_fullsized":"2024-04-25T23:52:56.907274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":4732,"log_dups_size":0,"ondisk_log_size":4732,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:43.611787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":89522176,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":1,"num_read_kb":64,"num_write":4706,"num_write_kb":131713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.304971+0000","last_change":"2024-04-25T23:48:58.906152+0000","last_active":"2024-04-25T23:52:42.304971+0000","last_peered":"2024-04-25T23:52:42.304971+0000","last_clean":"2024-04-25T23:52:42.304971+0000","last_became_active":"2024-04-25T23:48:58.905805+0000","last_became_peered":"2024-04-25T23:48:58.905805+0000","last_unstale":"2024-04-25T23:52:42.304971+0000","last_undegraded":"2024-04-25T23:52:42.304971+0000","last_fullsized":"2024-04-25T23:52:42.304971+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_clean_scrub_stamp":"2024-04-25T23:48:57.875228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:05:52.310323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"21'3162","reported_seq":3191,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.801171+0000","last_change":"2024-04-25T23:49:04.978319+0000","last_active":"2024-04-25T23:52:42.801171+0000","last_peered":"2024-04-25T23:52:42.801171+0000","last_clean":"2024-04-25T23:52:42.801171+0000","last_became_active":"2024-04-25T23:49:04.977360+0000","last_became_peered":"2024-04-25T23:49:04.977360+0000","last_unstale":"2024-04-25T23:52:42.801171+0000","last_undegraded":"2024-04-25T23:52:42.801171+0000","last_fullsized":"2024-04-25T23:52:42.801171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":3162,"log_dups_size":0,"ondisk_log_size":3162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:51:25.735100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96731136,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":55,"num_read_kb":103,"num_write":3148,"num_write_kb":156614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.300563+0000","last_change":"2024-04-25T23:48:59.122975+0000","last_active":"2024-04-25T23:52:42.300563+0000","last_peered":"2024-04-25T23:52:42.300563+0000","last_clean":"2024-04-25T23:52:42.300563+0000","last_became_active":"2024-04-25T23:48:59.122193+0000","last_became_peered":"2024-04-25T23:48:59.122193+0000","last_unstale":"2024-04-25T23:52:42.300563+0000","last_undegraded":"2024-04-25T23:52:42.300563+0000","last_fullsized":"2024-04-25T23:52:42.300563+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_clean_scrub_stamp":"2024-04-25T23:48:57.875228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:09:25.578965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"21'7247","reported_seq":7263,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:53:47.589729+0000","last_change":"2024-04-25T23:49:05.092246+0000","last_active":"2024-04-25T23:53:47.589729+0000","last_peered":"2024-04-25T23:53:47.589729+0000","last_clean":"2024-04-25T23:53:47.589729+0000","last_became_active":"2024-04-25T23:49:05.091503+0000","last_became_peered":"2024-04-25T23:49:05.091503+0000","last_unstale":"2024-04-25T23:53:47.589729+0000","last_undegraded":"2024-04-25T23:53:47.589729+0000","last_fullsized":"2024-04-25T23:53:47.589729+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":7247,"log_dups_size":0,"ondisk_log_size":7247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:41:54.323489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":140460032,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7239,"num_write_kb":212436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.304716+0000","last_change":"2024-04-25T23:48:59.120607+0000","last_active":"2024-04-25T23:52:42.304716+0000","last_peered":"2024-04-25T23:52:42.304716+0000","last_clean":"2024-04-25T23:52:42.304716+0000","last_became_active":"2024-04-25T23:48:59.119535+0000","last_became_peered":"2024-04-25T23:48:59.119535+0000","last_unstale":"2024-04-25T23:52:42.304716+0000","last_undegraded":"2024-04-25T23:52:42.304716+0000","last_fullsized":"2024-04-25T23:52:42.304716+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_clean_scrub_stamp":"2024-04-25T23:48:57.875228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:40:05.201442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"20'4309","reported_seq":4437,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.393106+0000","last_change":"2024-04-25T23:49:04.990547+0000","last_active":"2024-04-25T23:52:42.393106+0000","last_peered":"2024-04-25T23:52:42.393106+0000","last_clean":"2024-04-25T23:52:42.393106+0000","last_became_active":"2024-04-25T23:49:04.988031+0000","last_became_peered":"2024-04-25T23:49:04.988031+0000","last_unstale":"2024-04-25T23:52:42.393106+0000","last_undegraded":"2024-04-25T23:52:42.393106+0000","last_fullsized":"2024-04-25T23:52:42.393106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":4309,"log_dups_size":0,"ondisk_log_size":4309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:24:35.524086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100794896,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":148,"num_read_kb":139,"num_write":4335,"num_write_kb":170205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"20'3471","reported_seq":3487,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.320999+0000","last_change":"2024-04-25T23:49:04.995825+0000","last_active":"2024-04-25T23:52:42.320999+0000","last_peered":"2024-04-25T23:52:42.320999+0000","last_clean":"2024-04-25T23:52:42.320999+0000","last_became_active":"2024-04-25T23:49:04.995190+0000","last_became_peered":"2024-04-25T23:49:04.995190+0000","last_unstale":"2024-04-25T23:52:42.320999+0000","last_undegraded":"2024-04-25T23:52:42.320999+0000","last_fullsized":"2024-04-25T23:52:42.320999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":3471,"log_dups_size":0,"ondisk_log_size":3471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:23:45.839121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113246208,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3469,"num_write_kb":172802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.391698+0000","last_change":"2024-04-25T23:48:58.920072+0000","last_active":"2024-04-25T23:52:42.391698+0000","last_peered":"2024-04-25T23:52:42.391698+0000","last_clean":"2024-04-25T23:52:42.391698+0000","last_became_active":"2024-04-25T23:48:58.919554+0000","last_became_peered":"2024-04-25T23:48:58.919554+0000","last_unstale":"2024-04-25T23:52:42.391698+0000","last_undegraded":"2024-04-25T23:52:42.391698+0000","last_fullsized":"2024-04-25T23:52:42.391698+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_clean_scrub_stamp":"2024-04-25T23:48:57.875228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:47:42.230724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"21'3178","reported_seq":3208,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.801745+0000","last_change":"2024-04-25T23:49:05.092897+0000","last_active":"2024-04-25T23:52:42.801745+0000","last_peered":"2024-04-25T23:52:42.801745+0000","last_clean":"2024-04-25T23:52:42.801745+0000","last_became_active":"2024-04-25T23:49:05.092081+0000","last_became_peered":"2024-04-25T23:49:05.092081+0000","last_unstale":"2024-04-25T23:52:42.801745+0000","last_undegraded":"2024-04-25T23:52:42.801745+0000","last_fullsized":"2024-04-25T23:52:42.801745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":3178,"log_dups_size":0,"ondisk_log_size":3178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:15:07.022713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104857600,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":35,"num_read_kb":153,"num_write":3168,"num_write_kb":148628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.326922+0000","last_change":"2024-04-25T23:48:58.910610+0000","last_active":"2024-04-25T23:52:42.326922+0000","last_peered":"2024-04-25T23:52:42.326922+0000","last_clean":"2024-04-25T23:52:42.326922+0000","last_became_active":"2024-04-25T23:48:58.910228+0000","last_became_peered":"2024-04-25T23:48:58.910228+0000","last_unstale":"2024-04-25T23:52:42.326922+0000","last_undegraded":"2024-04-25T23:52:42.326922+0000","last_fullsized":"2024-04-25T23:52:42.326922+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_clean_scrub_stamp":"2024-04-25T23:48:57.875228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:40:43.690806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"21'2866","reported_seq":2883,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.802732+0000","last_change":"2024-04-25T23:49:04.996197+0000","last_active":"2024-04-25T23:52:42.802732+0000","last_peered":"2024-04-25T23:52:42.802732+0000","last_clean":"2024-04-25T23:52:42.802732+0000","last_became_active":"2024-04-25T23:49:04.995711+0000","last_became_peered":"2024-04-25T23:49:04.995711+0000","last_unstale":"2024-04-25T23:52:42.802732+0000","last_undegraded":"2024-04-25T23:52:42.802732+0000","last_fullsized":"2024-04-25T23:52:42.802732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":2866,"log_dups_size":0,"ondisk_log_size":2866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:44:41.716855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":95095128,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":2846,"num_write_kb":142623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.299915+0000","last_change":"2024-04-25T23:48:58.913466+0000","last_active":"2024-04-25T23:52:42.299915+0000","last_peered":"2024-04-25T23:52:42.299915+0000","last_clean":"2024-04-25T23:52:42.299915+0000","last_became_active":"2024-04-25T23:48:58.912966+0000","last_became_peered":"2024-04-25T23:48:58.912966+0000","last_unstale":"2024-04-25T23:52:42.299915+0000","last_undegraded":"2024-04-25T23:52:42.299915+0000","last_fullsized":"2024-04-25T23:52:42.299915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_clean_scrub_stamp":"2024-04-25T23:48:57.875228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:25:56.738346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"21'4357","reported_seq":4378,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:57.666350+0000","last_change":"2024-04-25T23:49:04.977221+0000","last_active":"2024-04-25T23:52:57.666350+0000","last_peered":"2024-04-25T23:52:57.666350+0000","last_clean":"2024-04-25T23:52:57.666350+0000","last_became_active":"2024-04-25T23:49:04.976471+0000","last_became_peered":"2024-04-25T23:49:04.976471+0000","last_unstale":"2024-04-25T23:52:57.666350+0000","last_undegraded":"2024-04-25T23:52:57.666350+0000","last_fullsized":"2024-04-25T23:52:57.666350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":4357,"log_dups_size":0,"ondisk_log_size":4357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:16:51.256351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88211456,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":22,"num_read_kb":16,"num_write":4360,"num_write_kb":167041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.391498+0000","last_change":"2024-04-25T23:48:58.915463+0000","last_active":"2024-04-25T23:52:42.391498+0000","last_peered":"2024-04-25T23:52:42.391498+0000","last_clean":"2024-04-25T23:52:42.391498+0000","last_became_active":"2024-04-25T23:48:58.914379+0000","last_became_peered":"2024-04-25T23:48:58.914379+0000","last_unstale":"2024-04-25T23:52:42.391498+0000","last_undegraded":"2024-04-25T23:52:42.391498+0000","last_fullsized":"2024-04-25T23:52:42.391498+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_clean_scrub_stamp":"2024-04-25T23:48:57.875228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:11:09.852719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":172,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.300390+0000","last_change":"2024-04-25T23:48:59.124853+0000","last_active":"2024-04-25T23:52:42.300390+0000","last_peered":"2024-04-25T23:52:42.300390+0000","last_clean":"2024-04-25T23:52:42.300390+0000","last_became_active":"2024-04-25T23:48:59.122379+0000","last_became_peered":"2024-04-25T23:48:59.122379+0000","last_unstale":"2024-04-25T23:52:42.300390+0000","last_undegraded":"2024-04-25T23:52:42.300390+0000","last_fullsized":"2024-04-25T23:52:42.300390+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_clean_scrub_stamp":"2024-04-25T23:48:57.875228+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:57:31.186906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"21'3897","reported_seq":3938,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.802677+0000","last_change":"2024-04-25T23:49:04.980999+0000","last_active":"2024-04-25T23:52:42.802677+0000","last_peered":"2024-04-25T23:52:42.802677+0000","last_clean":"2024-04-25T23:52:42.802677+0000","last_became_active":"2024-04-25T23:49:04.980272+0000","last_became_peered":"2024-04-25T23:49:04.980272+0000","last_unstale":"2024-04-25T23:52:42.802677+0000","last_undegraded":"2024-04-25T23:52:42.802677+0000","last_fullsized":"2024-04-25T23:52:42.802677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":3897,"log_dups_size":0,"ondisk_log_size":3897,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:43:10.867606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100794368,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":94,"num_read_kb":77,"num_write":3859,"num_write_kb":201602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"21'7829","reported_seq":7966,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:53:57.613005+0000","last_change":"2024-04-25T23:49:05.094703+0000","last_active":"2024-04-25T23:53:57.613005+0000","last_peered":"2024-04-25T23:53:57.613005+0000","last_clean":"2024-04-25T23:53:57.613005+0000","last_became_active":"2024-04-25T23:49:05.094007+0000","last_became_peered":"2024-04-25T23:49:05.094007+0000","last_unstale":"2024-04-25T23:53:57.613005+0000","last_undegraded":"2024-04-25T23:53:57.613005+0000","last_fullsized":"2024-04-25T23:53:57.613005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":7829,"log_dups_size":0,"ondisk_log_size":7829,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:33:21.090339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":95027712,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":414,"num_read_kb":363,"num_write":5838,"num_write_kb":183309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"21'4056","reported_seq":4079,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:53:47.643827+0000","last_change":"2024-04-25T23:49:04.983367+0000","last_active":"2024-04-25T23:53:47.643827+0000","last_peered":"2024-04-25T23:53:47.643827+0000","last_clean":"2024-04-25T23:53:47.643827+0000","last_became_active":"2024-04-25T23:49:04.982660+0000","last_became_peered":"2024-04-25T23:49:04.982660+0000","last_unstale":"2024-04-25T23:53:47.643827+0000","last_undegraded":"2024-04-25T23:53:47.643827+0000","last_fullsized":"2024-04-25T23:53:47.643827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":4056,"log_dups_size":0,"ondisk_log_size":4056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:34:08.771639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123396624,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":13,"num_read_kb":11,"num_write":4049,"num_write_kb":164094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"20'3795","reported_seq":3811,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.302007+0000","last_change":"2024-04-25T23:49:04.991004+0000","last_active":"2024-04-25T23:52:42.302007+0000","last_peered":"2024-04-25T23:52:42.302007+0000","last_clean":"2024-04-25T23:52:42.302007+0000","last_became_active":"2024-04-25T23:49:04.990144+0000","last_became_peered":"2024-04-25T23:49:04.990144+0000","last_unstale":"2024-04-25T23:52:42.302007+0000","last_undegraded":"2024-04-25T23:52:42.302007+0000","last_fullsized":"2024-04-25T23:52:42.302007+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":3795,"log_dups_size":0,"ondisk_log_size":3795,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:18:20.437365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113246208,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3792,"num_write_kb":169093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"21'3586","reported_seq":3613,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.794253+0000","last_change":"2024-04-25T23:49:04.988518+0000","last_active":"2024-04-25T23:52:42.794253+0000","last_peered":"2024-04-25T23:52:42.794253+0000","last_clean":"2024-04-25T23:52:42.794253+0000","last_became_active":"2024-04-25T23:49:04.987663+0000","last_became_peered":"2024-04-25T23:49:04.987663+0000","last_unstale":"2024-04-25T23:52:42.794253+0000","last_undegraded":"2024-04-25T23:52:42.794253+0000","last_fullsized":"2024-04-25T23:52:42.794253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":3586,"log_dups_size":0,"ondisk_log_size":3586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:08:28.379346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112721920,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":45,"num_read_kb":34,"num_write":3560,"num_write_kb":152196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"21'4479","reported_seq":4500,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.800823+0000","last_change":"2024-04-25T23:49:04.997861+0000","last_active":"2024-04-25T23:52:42.800823+0000","last_peered":"2024-04-25T23:52:42.800823+0000","last_clean":"2024-04-25T23:52:42.800823+0000","last_became_active":"2024-04-25T23:49:04.996781+0000","last_became_peered":"2024-04-25T23:49:04.996781+0000","last_unstale":"2024-04-25T23:52:42.800823+0000","last_undegraded":"2024-04-25T23:52:42.800823+0000","last_fullsized":"2024-04-25T23:52:42.800823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":4479,"log_dups_size":0,"ondisk_log_size":4479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:48:47.865840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":127338152,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":8,"num_read_kb":7,"num_write":4466,"num_write_kb":191484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"21'5770","reported_seq":5861,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.888218+0000","last_change":"2024-04-25T23:49:04.998785+0000","last_active":"2024-04-25T23:52:42.888218+0000","last_peered":"2024-04-25T23:52:42.888218+0000","last_clean":"2024-04-25T23:52:42.888218+0000","last_became_active":"2024-04-25T23:49:04.997977+0000","last_became_peered":"2024-04-25T23:49:04.997977+0000","last_unstale":"2024-04-25T23:52:42.888218+0000","last_undegraded":"2024-04-25T23:52:42.888218+0000","last_fullsized":"2024-04-25T23:52:42.888218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":5770,"log_dups_size":0,"ondisk_log_size":5770,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:13:53.271710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":116271984,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":286,"num_read_kb":214,"num_write":6036,"num_write_kb":177360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"21'3327","reported_seq":3343,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.804453+0000","last_change":"2024-04-25T23:49:04.998698+0000","last_active":"2024-04-25T23:52:42.804453+0000","last_peered":"2024-04-25T23:52:42.804453+0000","last_clean":"2024-04-25T23:52:42.804453+0000","last_became_active":"2024-04-25T23:49:04.997986+0000","last_became_peered":"2024-04-25T23:49:04.997986+0000","last_unstale":"2024-04-25T23:52:42.804453+0000","last_undegraded":"2024-04-25T23:52:42.804453+0000","last_fullsized":"2024-04-25T23:52:42.804453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":3327,"log_dups_size":0,"ondisk_log_size":3327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:59:11.386821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663296,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3277,"num_write_kb":152452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"21'3069","reported_seq":3085,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.796063+0000","last_change":"2024-04-25T23:49:04.990446+0000","last_active":"2024-04-25T23:52:42.796063+0000","last_peered":"2024-04-25T23:52:42.796063+0000","last_clean":"2024-04-25T23:52:42.796063+0000","last_became_active":"2024-04-25T23:49:04.989709+0000","last_became_peered":"2024-04-25T23:49:04.989709+0000","last_unstale":"2024-04-25T23:52:42.796063+0000","last_undegraded":"2024-04-25T23:52:42.796063+0000","last_fullsized":"2024-04-25T23:52:42.796063+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":3069,"log_dups_size":0,"ondisk_log_size":3069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:22:01.360709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3021,"num_write_kb":144132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"20'4577","reported_seq":4601,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.299824+0000","last_change":"2024-04-25T23:49:04.998536+0000","last_active":"2024-04-25T23:52:42.299824+0000","last_peered":"2024-04-25T23:52:42.299824+0000","last_clean":"2024-04-25T23:52:42.299824+0000","last_became_active":"2024-04-25T23:49:04.997718+0000","last_became_peered":"2024-04-25T23:49:04.997718+0000","last_unstale":"2024-04-25T23:52:42.299824+0000","last_undegraded":"2024-04-25T23:52:42.299824+0000","last_fullsized":"2024-04-25T23:52:42.299824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":4577,"log_dups_size":0,"ondisk_log_size":4577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:42:48.145557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117440512,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":4584,"num_write_kb":172941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"21'4926","reported_seq":4942,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.806464+0000","last_change":"2024-04-25T23:49:04.988912+0000","last_active":"2024-04-25T23:52:42.806464+0000","last_peered":"2024-04-25T23:52:42.806464+0000","last_clean":"2024-04-25T23:52:42.806464+0000","last_became_active":"2024-04-25T23:49:04.987367+0000","last_became_peered":"2024-04-25T23:49:04.987367+0000","last_unstale":"2024-04-25T23:52:42.806464+0000","last_undegraded":"2024-04-25T23:52:42.806464+0000","last_fullsized":"2024-04-25T23:52:42.806464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":4926,"log_dups_size":0,"ondisk_log_size":4926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:11:40.900415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":149553152,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4881,"num_write_kb":166152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"21'7343","reported_seq":7369,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:55.941996+0000","last_change":"2024-04-25T23:49:04.997739+0000","last_active":"2024-04-25T23:52:55.941996+0000","last_peered":"2024-04-25T23:52:55.941996+0000","last_clean":"2024-04-25T23:52:55.941996+0000","last_became_active":"2024-04-25T23:49:04.996841+0000","last_became_peered":"2024-04-25T23:49:04.996841+0000","last_unstale":"2024-04-25T23:52:55.941996+0000","last_undegraded":"2024-04-25T23:52:55.941996+0000","last_fullsized":"2024-04-25T23:52:55.941996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":7343,"log_dups_size":0,"ondisk_log_size":7343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:08:44.872965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83346448,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":33,"num_read_kb":88,"num_write":7349,"num_write_kb":122645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":3560846088,"num_objects":877,"num_object_clones":0,"num_object_copies":1754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":877,"num_whiteouts":0,"num_read":2507,"num_read_kb":2518,"num_write":155692,"num_write_kb":5620473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":6782795776,"data_stored":6820341912,"data_compressed":194288578,"data_compressed_allocated":202072064,"data_compressed_original":243892224,"omap_allocated":0,"internal_metadata":0},"log_size":157484,"ondisk_log_size":157484,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":12118,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542206,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":388020,"kb_used_data":298604,"kb_used_omap":1,"kb_used_meta":89406,"kb_avail":93983820,"statfs":{"total":96636764160,"available":96239431680,"internally_reserved":0,"allocated":305770496,"data_stored":306544866,"data_compressed":6553827,"data_compressed_allocated":6823936,"data_compressed_original":7905280,"omap_allocated":1593,"internal_metadata":91552199},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542208,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1212816,"kb_used_data":929032,"kb_used_omap":1,"kb_used_meta":283774,"kb_avail":93159024,"statfs":{"total":96636764160,"available":95394840576,"internally_reserved":0,"allocated":951328768,"data_stored":954362994,"data_compressed":23747576,"data_compressed_allocated":24735744,"data_compressed_original":28565504,"omap_allocated":1595,"internal_metadata":290585029},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542206,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1859000,"kb_used_data":1332336,"kb_used_omap":1,"kb_used_meta":526654,"kb_avail":92512840,"statfs":{"total":96636764160,"available":94733148160,"internally_reserved":0,"allocated":1364312064,"data_stored":1378232810,"data_compressed":53769970,"data_compressed_allocated":55988224,"data_compressed_original":70811648,"omap_allocated":1595,"internal_metadata":539294149},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574910,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1460108,"kb_used_data":1031236,"kb_used_omap":1,"kb_used_meta":428862,"kb_avail":92911732,"statfs":{"total":96636764160,"available":95141613568,"internally_reserved":0,"allocated":1055985664,"data_stored":1063157334,"data_compressed":33533197,"data_compressed_allocated":34906112,"data_compressed_original":43012096,"omap_allocated":1590,"internal_metadata":439155146},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574911,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1418012,"kb_used_data":1062036,"kb_used_omap":1,"kb_used_meta":355966,"kb_avail":92953828,"statfs":{"total":96636764160,"available":95184719872,"internally_reserved":0,"allocated":1087524864,"data_stored":1089954434,"data_compressed":21531874,"data_compressed_allocated":22360064,"data_compressed_original":25559040,"omap_allocated":1593,"internal_metadata":364509639},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574910,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":424748,"kb_used_data":303332,"kb_used_omap":1,"kb_used_meta":121406,"kb_avail":93947092,"statfs":{"total":96636764160,"available":96201822208,"internally_reserved":0,"allocated":310611968,"data_stored":314944206,"data_compressed":11229575,"data_compressed_allocated":11784192,"data_compressed_original":16420864,"omap_allocated":1595,"internal_metadata":124320197},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":861340,"kb_used_data":623380,"kb_used_omap":1,"kb_used_meta":237950,"kb_avail":93510500,"statfs":{"total":96636764160,"available":95754752000,"internally_reserved":0,"allocated":638341120,"data_stored":641442682,"data_compressed":20262986,"data_compressed_allocated":21012480,"data_compressed_original":24612864,"omap_allocated":1595,"internal_metadata":243661253},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574910,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1391756,"kb_used_data":1045508,"kb_used_omap":1,"kb_used_meta":346238,"kb_avail":92980084,"statfs":{"total":96636764160,"available":95211606016,"internally_reserved":0,"allocated":1070600192,"data_stored":1073278922,"data_compressed":23671691,"data_compressed_allocated":24526848,"data_compressed_original":28028928,"omap_allocated":1595,"internal_metadata":354548165},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":6059,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":6059,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1070350336,"data_stored":1072685184,"data_compressed":23665632,"data_compressed_allocated":24494080,"data_compressed_original":27516928,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":638144512,"data_stored":641377872,"data_compressed":20262986,"data_compressed_allocated":21012480,"data_compressed_original":24612864,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":310415360,"data_stored":314879396,"data_compressed":11229575,"data_compressed_allocated":11784192,"data_compressed_original":16420864,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1087328256,"data_stored":1089889624,"data_compressed":21531874,"data_compressed_allocated":22360064,"data_compressed_original":25559040,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1364115456,"data_stored":1378168000,"data_compressed":53769970,"data_compressed_allocated":55988224,"data_compressed_original":70811648,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":951132160,"data_stored":954298184,"data_compressed":23747576,"data_compressed_allocated":24735744,"data_compressed_original":28565504,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":305573888,"data_stored":306480056,"data_compressed":6553827,"data_compressed_allocated":6823936,"data_compressed_original":7905280,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1055735808,"data_stored":1062563596,"data_compressed":33527138,"data_compressed_allocated":34873344,"data_compressed_original":42500096,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T23:53:59.519 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T23:53:59.519 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T23:53:59.648 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.664+0000 7fe2f6666640 1 Processor -- start 2024-04-25T23:53:59.648 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.664+0000 7fe2f6666640 1 -- start start 2024-04-25T23:53:59.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.664+0000 7fe2f6666640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe2f00b2420 0x7fe2f01417c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:53:59.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.664+0000 7fe2f6666640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2f00b12a0 con 0x7fe2f00b2040 2024-04-25T23:53:59.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.664+0000 7fe2f6666640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2f0141d00 con 0x7fe2f01400a0 2024-04-25T23:53:59.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.664+0000 7fe2f6666640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fe2f00b2970 con 0x7fe2f00b2420 2024-04-25T23:53:59.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.664+0000 7fe2effff640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fe2f00b2040 0x7fe2f00b4ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:35354/0 (socket says 172.21.15.57:35354) 2024-04-25T23:53:59.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.664+0000 7fe2effff640 1 -- 172.21.15.57:0/3022510034 learned_addr learned my addr 172.21.15.57:0/3022510034 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-04-25T23:53:59.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.664+0000 7fe2ef7fe640 1 --2- 172.21.15.57:0/3022510034 >> v2:172.21.15.80:3300/0 conn(0x7fe2f00b2420 0x7fe2f01417c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:53:59.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.664+0000 7fe2eeffd640 1 -- 172.21.15.57:0/3022510034 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3119388945 0 0) 0x7fe2f00b12a0 con 0x7fe2f00b2040 2024-04-25T23:53:59.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.664+0000 7fe2eeffd640 1 -- 172.21.15.57:0/3022510034 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe2d8003530 con 0x7fe2f00b2040 2024-04-25T23:53:59.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.664+0000 7fe2eeffd640 1 -- 172.21.15.57:0/3022510034 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1643560174 0 0) 0x7fe2d8003530 con 0x7fe2f00b2040 2024-04-25T23:53:59.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.664+0000 7fe2eeffd640 1 -- 172.21.15.57:0/3022510034 >> v2:172.21.15.80:3300/0 conn(0x7fe2f00b2420 msgr2=0x7fe2f01417c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:53:59.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.664+0000 7fe2eeffd640 1 --2- 172.21.15.57:0/3022510034 >> v2:172.21.15.80:3300/0 conn(0x7fe2f00b2420 0x7fe2f01417c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:53:59.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.664+0000 7fe2eeffd640 1 -- 172.21.15.57:0/3022510034 >> v1:172.21.15.80:6789/0 conn(0x7fe2f01400a0 legacy=0x7fe2f01404b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:53:59.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.664+0000 7fe2eeffd640 1 -- 172.21.15.57:0/3022510034 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2f00b17f0 con 0x7fe2f00b2040 2024-04-25T23:53:59.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.664+0000 7fe2eeffd640 1 -- 172.21.15.57:0/3022510034 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2627952221 0 0) 0x7fe2e8001680 con 0x7fe2f00b2040 2024-04-25T23:53:59.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.664+0000 7fe2eeffd640 1 -- 172.21.15.57:0/3022510034 >> v1:172.21.15.57:6789/0 conn(0x7fe2f00b2040 legacy=0x7fe2f00b4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:53:59.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.664+0000 7fe2eeffd640 1 --2- 172.21.15.57:0/3022510034 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe2d80039c0 0x7fe2d8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:53:59.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.664+0000 7fe2eeffd640 1 --2- 172.21.15.57:0/3022510034 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe2d8004810 0x7fe2d8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:53:59.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.664+0000 7fe2eeffd640 1 --2- 172.21.15.57:0/3022510034 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fe2d80051d0 0x7fe2d800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:53:59.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.664+0000 7fe2eeffd640 1 -- 172.21.15.57:0/3022510034 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f00b2970 con 0x7fe2d8004810 2024-04-25T23:53:59.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.664+0000 7fe2eeffd640 1 -- 172.21.15.57:0/3022510034 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d800c170 con 0x7fe2d80039c0 2024-04-25T23:53:59.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.664+0000 7fe2eeffd640 1 -- 172.21.15.57:0/3022510034 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7fe2d800c370 con 0x7fe2d80051d0 2024-04-25T23:53:59.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.664+0000 7fe2ef7fe640 1 --2- 172.21.15.57:0/3022510034 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe2d8004810 0x7fe2d8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:53:59.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.664+0000 7fe2f4c03640 1 --2- 172.21.15.57:0/3022510034 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fe2d80051d0 0x7fe2d800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:53:59.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.664+0000 7fe2effff640 1 --2- 172.21.15.57:0/3022510034 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe2d80039c0 0x7fe2d8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:53:59.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.664+0000 7fe2ef7fe640 1 -- 172.21.15.57:0/3022510034 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fe2d80051d0 msgr2=0x7fe2d800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:53:59.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.664+0000 7fe2ef7fe640 1 --2- 172.21.15.57:0/3022510034 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fe2d80051d0 0x7fe2d800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:53:59.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.664+0000 7fe2ef7fe640 1 -- 172.21.15.57:0/3022510034 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe2d80039c0 msgr2=0x7fe2d8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:53:59.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.664+0000 7fe2ef7fe640 1 --2- 172.21.15.57:0/3022510034 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe2d80039c0 0x7fe2d8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:53:59.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.664+0000 7fe2ef7fe640 1 -- 172.21.15.57:0/3022510034 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2d8003530 con 0x7fe2d8004810 2024-04-25T23:53:59.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.664+0000 7fe2f4c03640 1 -- 172.21.15.57:0/3022510034 reap_dead start 2024-04-25T23:53:59.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.668+0000 7fe2ef7fe640 1 --2- 172.21.15.57:0/3022510034 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe2d8004810 0x7fe2d8004c90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4009fc0 tx=0x7fe2e401cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6cef021fcb6c00 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:53:59.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.668+0000 7fe2eeffd640 1 -- 172.21.15.57:0/3022510034 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e40b2070 con 0x7fe2d8004810 2024-04-25T23:53:59.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.668+0000 7fe2eeffd640 1 -- 172.21.15.57:0/3022510034 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2e40a5b60 con 0x7fe2d8004810 2024-04-25T23:53:59.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.668+0000 7fe2eeffd640 1 -- 172.21.15.57:0/3022510034 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e4009c10 con 0x7fe2d8004810 2024-04-25T23:53:59.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.668+0000 7fe2f6666640 1 -- 172.21.15.57:0/3022510034 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe2d8004810 msgr2=0x7fe2d8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:53:59.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.668+0000 7fe2f6666640 1 --2- 172.21.15.57:0/3022510034 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe2d8004810 0x7fe2d8004c90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4009fc0 tx=0x7fe2e401cc60 comp rx=0 tx=0).stop 2024-04-25T23:53:59.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.668+0000 7fe2f6666640 1 -- 172.21.15.57:0/3022510034 shutdown_connections 2024-04-25T23:53:59.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.668+0000 7fe2f6666640 1 --2- 172.21.15.57:0/3022510034 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe2d8004810 0x7fe2d8004c90 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:53:59.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.668+0000 7fe2f6666640 1 -- 172.21.15.57:0/3022510034 >> 172.21.15.57:0/3022510034 conn(0x7fe2f00b6db0 msgr2=0x7fe2f00b71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:53:59.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.668+0000 7fe2f6666640 1 -- 172.21.15.57:0/3022510034 shutdown_connections 2024-04-25T23:53:59.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.668+0000 7fe2f6666640 1 -- 172.21.15.57:0/3022510034 wait complete. 2024-04-25T23:53:59.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.668+0000 7fe2f6666640 1 Processor -- start 2024-04-25T23:53:59.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.668+0000 7fe2f6666640 1 -- start start 2024-04-25T23:53:59.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.672+0000 7fe2f6666640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe2d8004810 0x7fe2f01ced30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:53:59.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.672+0000 7fe2f6666640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fe2f013c510 0x7fe2f013c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:53:59.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.672+0000 7fe2f6666640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe2f0137330 0x7fe2f0137770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:53:59.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.672+0000 7fe2f6666640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f00b4e70 con 0x7fe2d8004810 2024-04-25T23:53:59.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.672+0000 7fe2f6666640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f00b4cf0 con 0x7fe2f0137330 2024-04-25T23:53:59.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.672+0000 7fe2f6666640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7fe2f00b4ff0 con 0x7fe2f013c510 2024-04-25T23:53:59.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.672+0000 7fe2effff640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe2d8004810 0x7fe2f01ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:53:59.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.672+0000 7fe2ef7fe640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fe2f013c510 0x7fe2f013c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:53:59.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.672+0000 7fe2effff640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe2d8004810 0x7fe2f01ced30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37136/0 (socket says 172.21.15.57:37136) 2024-04-25T23:53:59.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.672+0000 7fe2effff640 1 -- 172.21.15.57:0/119978944 learned_addr learned my addr 172.21.15.57:0/119978944 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:53:59.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.672+0000 7fe2f4c03640 1 --2- 172.21.15.57:0/119978944 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe2f0137330 0x7fe2f0137770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:53:59.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.672+0000 7fe2effff640 1 -- 172.21.15.57:0/119978944 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fe2f013c510 msgr2=0x7fe2f013c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:53:59.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.672+0000 7fe2effff640 1 --2- 172.21.15.57:0/119978944 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fe2f013c510 0x7fe2f013c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:53:59.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.672+0000 7fe2effff640 1 -- 172.21.15.57:0/119978944 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe2f0137330 msgr2=0x7fe2f0137770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:53:59.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.672+0000 7fe2effff640 1 --2- 172.21.15.57:0/119978944 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe2f0137330 0x7fe2f0137770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:53:59.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.672+0000 7fe2effff640 1 -- 172.21.15.57:0/119978944 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2f0135f50 con 0x7fe2d8004810 2024-04-25T23:53:59.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.672+0000 7fe2effff640 1 --2- 172.21.15.57:0/119978944 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe2d8004810 0x7fe2f01ced30 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe2e800e7b0 tx=0x7fe2e800ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:53:59.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.672+0000 7fe2ecff9640 1 -- 172.21.15.57:0/119978944 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e800c6c0 con 0x7fe2d8004810 2024-04-25T23:53:59.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.672+0000 7fe2ecff9640 1 -- 172.21.15.57:0/119978944 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2e8018070 con 0x7fe2d8004810 2024-04-25T23:53:59.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.672+0000 7fe2f6666640 1 -- 172.21.15.57:0/119978944 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2f0139780 con 0x7fe2d8004810 2024-04-25T23:53:59.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.672+0000 7fe2ecff9640 1 -- 172.21.15.57:0/119978944 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e800cac0 con 0x7fe2d8004810 2024-04-25T23:53:59.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.672+0000 7fe2f6666640 1 -- 172.21.15.57:0/119978944 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2f0137cb0 con 0x7fe2d8004810 2024-04-25T23:53:59.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.672+0000 7fe2f6666640 1 -- 172.21.15.57:0/119978944 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2f01da650 con 0x7fe2d8004810 2024-04-25T23:53:59.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.672+0000 7fe2ecff9640 1 -- 172.21.15.57:0/119978944 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) ==== 87410+0+0 (secure 0 0 0) 0x7fe2e800cc60 con 0x7fe2d8004810 2024-04-25T23:53:59.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.672+0000 7fe2ecff9640 1 --2- 172.21.15.57:0/119978944 >> v2:172.21.15.80:6816/2493457677 conn(0x7fe2dc0678b0 0x7fe2dc069d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:53:59.663 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.672+0000 7fe2ecff9640 1 -- 172.21.15.57:0/119978944 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 4525+0+0 (secure 0 0 0) 0x7fe2e80851d0 con 0x7fe2d8004810 2024-04-25T23:53:59.663 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.672+0000 7fe2ef7fe640 1 --2- 172.21.15.57:0/119978944 >> v2:172.21.15.80:6816/2493457677 conn(0x7fe2dc0678b0 0x7fe2dc069d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:53:59.663 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.676+0000 7fe2ef7fe640 1 --2- 172.21.15.57:0/119978944 >> v2:172.21.15.80:6816/2493457677 conn(0x7fe2dc0678b0 0x7fe2dc069d70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe2e40af000 tx=0x7fe2e40a7040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:53:59.663 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.680+0000 7fe2ecff9640 1 -- 172.21.15.57:0/119978944 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe2e8056240 con 0x7fe2d8004810 2024-04-25T23:53:59.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.836+0000 7fe2f6666640 1 -- 172.21.15.57:0/119978944 --> v2:172.21.15.80:6816/2493457677 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe2f01cf530 con 0x7fe2dc0678b0 2024-04-25T23:53:59.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.844+0000 7fe2ecff9640 1 -- 172.21.15.57:0/119978944 <== mgr.4100 v2:172.21.15.80:6816/2493457677 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+117689 (secure 0 0 0) 0x7fe2e4009c10 con 0x7fe2dc0678b0 2024-04-25T23:53:59.832 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-25T23:53:59.833 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2024-04-25T23:53:59.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.848+0000 7fe2f6666640 1 -- 172.21.15.57:0/119978944 >> v2:172.21.15.80:6816/2493457677 conn(0x7fe2dc0678b0 msgr2=0x7fe2dc069d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:53:59.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.848+0000 7fe2f6666640 1 --2- 172.21.15.57:0/119978944 >> v2:172.21.15.80:6816/2493457677 conn(0x7fe2dc0678b0 0x7fe2dc069d70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe2e40af000 tx=0x7fe2e40a7040 comp rx=0 tx=0).stop 2024-04-25T23:53:59.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.848+0000 7fe2f6666640 1 -- 172.21.15.57:0/119978944 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe2d8004810 msgr2=0x7fe2f01ced30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:53:59.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.848+0000 7fe2f6666640 1 --2- 172.21.15.57:0/119978944 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe2d8004810 0x7fe2f01ced30 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe2e800e7b0 tx=0x7fe2e800ebb0 comp rx=0 tx=0).stop 2024-04-25T23:53:59.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.848+0000 7fe2f6666640 1 -- 172.21.15.57:0/119978944 shutdown_connections 2024-04-25T23:53:59.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.848+0000 7fe2f6666640 1 --2- 172.21.15.57:0/119978944 >> v2:172.21.15.80:6816/2493457677 conn(0x7fe2dc0678b0 0x7fe2dc069d70 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:53:59.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.848+0000 7fe2f6666640 1 --2- 172.21.15.57:0/119978944 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe2f0137330 0x7fe2f0137770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:53:59.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.848+0000 7fe2f6666640 1 --2- 172.21.15.57:0/119978944 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fe2f013c510 0x7fe2f013c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:53:59.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.848+0000 7fe2f6666640 1 --2- 172.21.15.57:0/119978944 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe2d8004810 0x7fe2f01ced30 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:53:59.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.848+0000 7fe2f6666640 1 -- 172.21.15.57:0/119978944 >> 172.21.15.57:0/119978944 conn(0x7fe2f00b6db0 msgr2=0x7fe2f00b7190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:53:59.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.848+0000 7fe2f6666640 1 -- 172.21.15.57:0/119978944 shutdown_connections 2024-04-25T23:53:59.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.848+0000 7fe2f6666640 1 -- 172.21.15.57:0/119978944 wait complete. 2024-04-25T23:53:59.849 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":166,"stamp":"2024-04-25T23:53:58.919008+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3561436456,"num_objects":879,"num_object_clones":0,"num_object_copies":1758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":879,"num_whiteouts":0,"num_read":2623,"num_read_kb":2618,"num_write":155851,"num_write_kb":5622889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157601,"ondisk_log_size":157601,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":9015800,"kb_used_data":6625464,"kb_used_omap":12,"kb_used_meta":2390259,"kb_avail":745958920,"statfs":{"total":773094113280,"available":763861934080,"internally_reserved":0,"allocated":6784475136,"data_stored":6821918248,"data_compressed":194300696,"data_compressed_allocated":202137600,"data_compressed_original":244916224,"omap_allocated":12751,"internal_metadata":2447625777},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":16384,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":5,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003287"},"pg_stats":[{"pgid":"3.1f","version":"20'4490","reported_seq":4529,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.383532+0000","last_change":"2024-04-25T23:49:04.991824+0000","last_active":"2024-04-25T23:52:42.383532+0000","last_peered":"2024-04-25T23:52:42.383532+0000","last_clean":"2024-04-25T23:52:42.383532+0000","last_became_active":"2024-04-25T23:49:04.991059+0000","last_became_peered":"2024-04-25T23:49:04.991059+0000","last_unstale":"2024-04-25T23:52:42.383532+0000","last_undegraded":"2024-04-25T23:52:42.383532+0000","last_fullsized":"2024-04-25T23:52:42.383532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":4490,"log_dups_size":0,"ondisk_log_size":4490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:06:08.003629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121634816,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":93,"num_read_kb":81,"num_write":4513,"num_write_kb":218308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"21'5438","reported_seq":5465,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:50.093018+0000","last_change":"2024-04-25T23:49:04.991993+0000","last_active":"2024-04-25T23:52:50.093018+0000","last_peered":"2024-04-25T23:52:50.093018+0000","last_clean":"2024-04-25T23:52:50.093018+0000","last_became_active":"2024-04-25T23:49:04.991224+0000","last_became_peered":"2024-04-25T23:49:04.991224+0000","last_unstale":"2024-04-25T23:52:50.093018+0000","last_undegraded":"2024-04-25T23:52:50.093018+0000","last_fullsized":"2024-04-25T23:52:50.093018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":5438,"log_dups_size":0,"ondisk_log_size":5438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:49:45.384039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":45,"num_read_kb":37,"num_write":5439,"num_write_kb":172929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"21'6313","reported_seq":6337,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:56.683805+0000","last_change":"2024-04-25T23:49:04.984448+0000","last_active":"2024-04-25T23:52:56.683805+0000","last_peered":"2024-04-25T23:52:56.683805+0000","last_clean":"2024-04-25T23:52:56.683805+0000","last_became_active":"2024-04-25T23:49:04.982204+0000","last_became_peered":"2024-04-25T23:49:04.982204+0000","last_unstale":"2024-04-25T23:52:56.683805+0000","last_undegraded":"2024-04-25T23:52:56.683805+0000","last_fullsized":"2024-04-25T23:52:56.683805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":6313,"log_dups_size":0,"ondisk_log_size":6313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:23:55.891971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":150994944,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":30,"num_read_kb":85,"num_write":6420,"num_write_kb":204243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"20'3855","reported_seq":3874,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.391433+0000","last_change":"2024-04-25T23:49:04.980119+0000","last_active":"2024-04-25T23:52:42.391433+0000","last_peered":"2024-04-25T23:52:42.391433+0000","last_clean":"2024-04-25T23:52:42.391433+0000","last_became_active":"2024-04-25T23:49:04.979323+0000","last_became_peered":"2024-04-25T23:49:04.979323+0000","last_unstale":"2024-04-25T23:52:42.391433+0000","last_undegraded":"2024-04-25T23:52:42.391433+0000","last_fullsized":"2024-04-25T23:52:42.391433+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":3855,"log_dups_size":0,"ondisk_log_size":3855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:09:46.513536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96535048,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":17,"num_read_kb":12,"num_write":3868,"num_write_kb":155287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"21'4570","reported_seq":4585,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.802622+0000","last_change":"2024-04-25T23:49:05.002350+0000","last_active":"2024-04-25T23:52:42.802622+0000","last_peered":"2024-04-25T23:52:42.802622+0000","last_clean":"2024-04-25T23:52:42.802622+0000","last_became_active":"2024-04-25T23:49:05.001246+0000","last_became_peered":"2024-04-25T23:49:05.001246+0000","last_unstale":"2024-04-25T23:52:42.802622+0000","last_undegraded":"2024-04-25T23:52:42.802622+0000","last_fullsized":"2024-04-25T23:52:42.802622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":4570,"log_dups_size":0,"ondisk_log_size":4570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:27:02.668986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":107675648,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4523,"num_write_kb":223556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"21'6959","reported_seq":6974,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:50.115868+0000","last_change":"2024-04-25T23:49:04.984321+0000","last_active":"2024-04-25T23:52:50.115868+0000","last_peered":"2024-04-25T23:52:50.115868+0000","last_clean":"2024-04-25T23:52:50.115868+0000","last_became_active":"2024-04-25T23:49:04.983518+0000","last_became_peered":"2024-04-25T23:49:04.983518+0000","last_unstale":"2024-04-25T23:52:50.115868+0000","last_undegraded":"2024-04-25T23:52:50.115868+0000","last_fullsized":"2024-04-25T23:52:50.115868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":6959,"log_dups_size":0,"ondisk_log_size":6959,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:31:32.872285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":138412032,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6933,"num_write_kb":206597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"21'5764","reported_seq":5780,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:53:01.763608+0000","last_change":"2024-04-25T23:49:04.985251+0000","last_active":"2024-04-25T23:53:01.763608+0000","last_peered":"2024-04-25T23:53:01.763608+0000","last_clean":"2024-04-25T23:53:01.763608+0000","last_became_active":"2024-04-25T23:49:04.984378+0000","last_became_peered":"2024-04-25T23:49:04.984378+0000","last_unstale":"2024-04-25T23:53:01.763608+0000","last_undegraded":"2024-04-25T23:53:01.763608+0000","last_fullsized":"2024-04-25T23:53:01.763608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":5764,"log_dups_size":0,"ondisk_log_size":5764,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:43:34.630095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121634816,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5717,"num_write_kb":246662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"21'7903","reported_seq":7920,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:53:00.638258+0000","last_change":"2024-04-25T23:49:04.988246+0000","last_active":"2024-04-25T23:53:00.638258+0000","last_peered":"2024-04-25T23:53:00.638258+0000","last_clean":"2024-04-25T23:53:00.638258+0000","last_became_active":"2024-04-25T23:49:04.987468+0000","last_became_peered":"2024-04-25T23:49:04.987468+0000","last_unstale":"2024-04-25T23:53:00.638258+0000","last_undegraded":"2024-04-25T23:53:00.638258+0000","last_fullsized":"2024-04-25T23:53:00.638258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":7903,"log_dups_size":0,"ondisk_log_size":7903,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:21:42.911815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":130023936,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":7908,"num_write_kb":222727,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"21'5984","reported_seq":6008,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:53.277327+0000","last_change":"2024-04-25T23:49:04.989029+0000","last_active":"2024-04-25T23:52:53.277327+0000","last_peered":"2024-04-25T23:52:53.277327+0000","last_clean":"2024-04-25T23:52:53.277327+0000","last_became_active":"2024-04-25T23:49:04.987729+0000","last_became_peered":"2024-04-25T23:49:04.987729+0000","last_unstale":"2024-04-25T23:52:53.277327+0000","last_undegraded":"2024-04-25T23:52:53.277327+0000","last_fullsized":"2024-04-25T23:52:53.277327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":5984,"log_dups_size":0,"ondisk_log_size":5984,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:34:36.865614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121634816,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":5995,"num_write_kb":201733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"21'3557","reported_seq":3881,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.801711+0000","last_change":"2024-04-25T23:49:04.986698+0000","last_active":"2024-04-25T23:52:42.801711+0000","last_peered":"2024-04-25T23:52:42.801711+0000","last_clean":"2024-04-25T23:52:42.801711+0000","last_became_active":"2024-04-25T23:49:04.985957+0000","last_became_peered":"2024-04-25T23:49:04.985957+0000","last_unstale":"2024-04-25T23:52:42.801711+0000","last_undegraded":"2024-04-25T23:52:42.801711+0000","last_fullsized":"2024-04-25T23:52:42.801711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":3557,"log_dups_size":0,"ondisk_log_size":3557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:07:54.597105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":77201936,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":1028,"num_read_kb":921,"num_write":3751,"num_write_kb":130317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"21'7645","reported_seq":7661,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:53:00.534834+0000","last_change":"2024-04-25T23:49:04.992705+0000","last_active":"2024-04-25T23:53:00.534834+0000","last_peered":"2024-04-25T23:53:00.534834+0000","last_clean":"2024-04-25T23:53:00.534834+0000","last_became_active":"2024-04-25T23:49:04.990812+0000","last_became_peered":"2024-04-25T23:49:04.990812+0000","last_unstale":"2024-04-25T23:53:00.534834+0000","last_undegraded":"2024-04-25T23:53:00.534834+0000","last_fullsized":"2024-04-25T23:53:00.534834+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":7645,"log_dups_size":0,"ondisk_log_size":7645,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:27:48.304794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":138412032,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7636,"num_write_kb":198278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"21'5030","reported_seq":5062,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:52.943801+0000","last_change":"2024-04-25T23:49:04.990899+0000","last_active":"2024-04-25T23:52:52.943801+0000","last_peered":"2024-04-25T23:52:52.943801+0000","last_clean":"2024-04-25T23:52:52.943801+0000","last_became_active":"2024-04-25T23:49:04.988250+0000","last_became_peered":"2024-04-25T23:49:04.988250+0000","last_unstale":"2024-04-25T23:52:52.943801+0000","last_undegraded":"2024-04-25T23:52:52.943801+0000","last_fullsized":"2024-04-25T23:52:52.943801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":5030,"log_dups_size":0,"ondisk_log_size":5030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:53:15.377369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":95029072,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":68,"num_read_kb":57,"num_write":5006,"num_write_kb":140314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.385292+0000","last_change":"2024-04-25T23:48:58.911464+0000","last_active":"2024-04-25T23:52:42.385292+0000","last_peered":"2024-04-25T23:52:42.385292+0000","last_clean":"2024-04-25T23:52:42.385292+0000","last_became_active":"2024-04-25T23:48:58.911101+0000","last_became_peered":"2024-04-25T23:48:58.911101+0000","last_unstale":"2024-04-25T23:52:42.385292+0000","last_undegraded":"2024-04-25T23:52:42.385292+0000","last_fullsized":"2024-04-25T23:52:42.385292+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_clean_scrub_stamp":"2024-04-25T23:48:57.875228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:06:28.527623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"21'4732","reported_seq":4749,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:56.907274+0000","last_change":"2024-04-25T23:49:04.985258+0000","last_active":"2024-04-25T23:52:56.907274+0000","last_peered":"2024-04-25T23:52:56.907274+0000","last_clean":"2024-04-25T23:52:56.907274+0000","last_became_active":"2024-04-25T23:49:04.984516+0000","last_became_peered":"2024-04-25T23:49:04.984516+0000","last_unstale":"2024-04-25T23:52:56.907274+0000","last_undegraded":"2024-04-25T23:52:56.907274+0000","last_fullsized":"2024-04-25T23:52:56.907274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":4732,"log_dups_size":0,"ondisk_log_size":4732,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:43.611787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":89522176,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":1,"num_read_kb":64,"num_write":4706,"num_write_kb":131713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.304971+0000","last_change":"2024-04-25T23:48:58.906152+0000","last_active":"2024-04-25T23:52:42.304971+0000","last_peered":"2024-04-25T23:52:42.304971+0000","last_clean":"2024-04-25T23:52:42.304971+0000","last_became_active":"2024-04-25T23:48:58.905805+0000","last_became_peered":"2024-04-25T23:48:58.905805+0000","last_unstale":"2024-04-25T23:52:42.304971+0000","last_undegraded":"2024-04-25T23:52:42.304971+0000","last_fullsized":"2024-04-25T23:52:42.304971+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_clean_scrub_stamp":"2024-04-25T23:48:57.875228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:05:52.310323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"21'3162","reported_seq":3191,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.801171+0000","last_change":"2024-04-25T23:49:04.978319+0000","last_active":"2024-04-25T23:52:42.801171+0000","last_peered":"2024-04-25T23:52:42.801171+0000","last_clean":"2024-04-25T23:52:42.801171+0000","last_became_active":"2024-04-25T23:49:04.977360+0000","last_became_peered":"2024-04-25T23:49:04.977360+0000","last_unstale":"2024-04-25T23:52:42.801171+0000","last_undegraded":"2024-04-25T23:52:42.801171+0000","last_fullsized":"2024-04-25T23:52:42.801171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":3162,"log_dups_size":0,"ondisk_log_size":3162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:51:25.735100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96731136,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":55,"num_read_kb":103,"num_write":3148,"num_write_kb":156614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.300563+0000","last_change":"2024-04-25T23:48:59.122975+0000","last_active":"2024-04-25T23:52:42.300563+0000","last_peered":"2024-04-25T23:52:42.300563+0000","last_clean":"2024-04-25T23:52:42.300563+0000","last_became_active":"2024-04-25T23:48:59.122193+0000","last_became_peered":"2024-04-25T23:48:59.122193+0000","last_unstale":"2024-04-25T23:52:42.300563+0000","last_undegraded":"2024-04-25T23:52:42.300563+0000","last_fullsized":"2024-04-25T23:52:42.300563+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_clean_scrub_stamp":"2024-04-25T23:48:57.875228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:09:25.578965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"21'7247","reported_seq":7263,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:53:47.589729+0000","last_change":"2024-04-25T23:49:05.092246+0000","last_active":"2024-04-25T23:53:47.589729+0000","last_peered":"2024-04-25T23:53:47.589729+0000","last_clean":"2024-04-25T23:53:47.589729+0000","last_became_active":"2024-04-25T23:49:05.091503+0000","last_became_peered":"2024-04-25T23:49:05.091503+0000","last_unstale":"2024-04-25T23:53:47.589729+0000","last_undegraded":"2024-04-25T23:53:47.589729+0000","last_fullsized":"2024-04-25T23:53:47.589729+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":7247,"log_dups_size":0,"ondisk_log_size":7247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:41:54.323489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":140460032,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7239,"num_write_kb":212436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.304716+0000","last_change":"2024-04-25T23:48:59.120607+0000","last_active":"2024-04-25T23:52:42.304716+0000","last_peered":"2024-04-25T23:52:42.304716+0000","last_clean":"2024-04-25T23:52:42.304716+0000","last_became_active":"2024-04-25T23:48:59.119535+0000","last_became_peered":"2024-04-25T23:48:59.119535+0000","last_unstale":"2024-04-25T23:52:42.304716+0000","last_undegraded":"2024-04-25T23:52:42.304716+0000","last_fullsized":"2024-04-25T23:52:42.304716+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_clean_scrub_stamp":"2024-04-25T23:48:57.875228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:40:05.201442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"20'4309","reported_seq":4437,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.393106+0000","last_change":"2024-04-25T23:49:04.990547+0000","last_active":"2024-04-25T23:52:42.393106+0000","last_peered":"2024-04-25T23:52:42.393106+0000","last_clean":"2024-04-25T23:52:42.393106+0000","last_became_active":"2024-04-25T23:49:04.988031+0000","last_became_peered":"2024-04-25T23:49:04.988031+0000","last_unstale":"2024-04-25T23:52:42.393106+0000","last_undegraded":"2024-04-25T23:52:42.393106+0000","last_fullsized":"2024-04-25T23:52:42.393106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":4309,"log_dups_size":0,"ondisk_log_size":4309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:24:35.524086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100794896,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":148,"num_read_kb":139,"num_write":4335,"num_write_kb":170205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"20'3471","reported_seq":3487,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.320999+0000","last_change":"2024-04-25T23:49:04.995825+0000","last_active":"2024-04-25T23:52:42.320999+0000","last_peered":"2024-04-25T23:52:42.320999+0000","last_clean":"2024-04-25T23:52:42.320999+0000","last_became_active":"2024-04-25T23:49:04.995190+0000","last_became_peered":"2024-04-25T23:49:04.995190+0000","last_unstale":"2024-04-25T23:52:42.320999+0000","last_undegraded":"2024-04-25T23:52:42.320999+0000","last_fullsized":"2024-04-25T23:52:42.320999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":3471,"log_dups_size":0,"ondisk_log_size":3471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:23:45.839121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113246208,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3469,"num_write_kb":172802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.391698+0000","last_change":"2024-04-25T23:48:58.920072+0000","last_active":"2024-04-25T23:52:42.391698+0000","last_peered":"2024-04-25T23:52:42.391698+0000","last_clean":"2024-04-25T23:52:42.391698+0000","last_became_active":"2024-04-25T23:48:58.919554+0000","last_became_peered":"2024-04-25T23:48:58.919554+0000","last_unstale":"2024-04-25T23:52:42.391698+0000","last_undegraded":"2024-04-25T23:52:42.391698+0000","last_fullsized":"2024-04-25T23:52:42.391698+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_clean_scrub_stamp":"2024-04-25T23:48:57.875228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:47:42.230724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"21'3178","reported_seq":3208,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.801745+0000","last_change":"2024-04-25T23:49:05.092897+0000","last_active":"2024-04-25T23:52:42.801745+0000","last_peered":"2024-04-25T23:52:42.801745+0000","last_clean":"2024-04-25T23:52:42.801745+0000","last_became_active":"2024-04-25T23:49:05.092081+0000","last_became_peered":"2024-04-25T23:49:05.092081+0000","last_unstale":"2024-04-25T23:52:42.801745+0000","last_undegraded":"2024-04-25T23:52:42.801745+0000","last_fullsized":"2024-04-25T23:52:42.801745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":3178,"log_dups_size":0,"ondisk_log_size":3178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:15:07.022713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104857600,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":35,"num_read_kb":153,"num_write":3168,"num_write_kb":148628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.326922+0000","last_change":"2024-04-25T23:48:58.910610+0000","last_active":"2024-04-25T23:52:42.326922+0000","last_peered":"2024-04-25T23:52:42.326922+0000","last_clean":"2024-04-25T23:52:42.326922+0000","last_became_active":"2024-04-25T23:48:58.910228+0000","last_became_peered":"2024-04-25T23:48:58.910228+0000","last_unstale":"2024-04-25T23:52:42.326922+0000","last_undegraded":"2024-04-25T23:52:42.326922+0000","last_fullsized":"2024-04-25T23:52:42.326922+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_clean_scrub_stamp":"2024-04-25T23:48:57.875228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:40:43.690806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"21'2866","reported_seq":2883,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.802732+0000","last_change":"2024-04-25T23:49:04.996197+0000","last_active":"2024-04-25T23:52:42.802732+0000","last_peered":"2024-04-25T23:52:42.802732+0000","last_clean":"2024-04-25T23:52:42.802732+0000","last_became_active":"2024-04-25T23:49:04.995711+0000","last_became_peered":"2024-04-25T23:49:04.995711+0000","last_unstale":"2024-04-25T23:52:42.802732+0000","last_undegraded":"2024-04-25T23:52:42.802732+0000","last_fullsized":"2024-04-25T23:52:42.802732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":2866,"log_dups_size":0,"ondisk_log_size":2866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:44:41.716855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":95095128,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":2846,"num_write_kb":142623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.299915+0000","last_change":"2024-04-25T23:48:58.913466+0000","last_active":"2024-04-25T23:52:42.299915+0000","last_peered":"2024-04-25T23:52:42.299915+0000","last_clean":"2024-04-25T23:52:42.299915+0000","last_became_active":"2024-04-25T23:48:58.912966+0000","last_became_peered":"2024-04-25T23:48:58.912966+0000","last_unstale":"2024-04-25T23:52:42.299915+0000","last_undegraded":"2024-04-25T23:52:42.299915+0000","last_fullsized":"2024-04-25T23:52:42.299915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_clean_scrub_stamp":"2024-04-25T23:48:57.875228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:25:56.738346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"21'4357","reported_seq":4378,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:57.666350+0000","last_change":"2024-04-25T23:49:04.977221+0000","last_active":"2024-04-25T23:52:57.666350+0000","last_peered":"2024-04-25T23:52:57.666350+0000","last_clean":"2024-04-25T23:52:57.666350+0000","last_became_active":"2024-04-25T23:49:04.976471+0000","last_became_peered":"2024-04-25T23:49:04.976471+0000","last_unstale":"2024-04-25T23:52:57.666350+0000","last_undegraded":"2024-04-25T23:52:57.666350+0000","last_fullsized":"2024-04-25T23:52:57.666350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":4357,"log_dups_size":0,"ondisk_log_size":4357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:16:51.256351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88211456,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":22,"num_read_kb":16,"num_write":4360,"num_write_kb":167041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.391498+0000","last_change":"2024-04-25T23:48:58.915463+0000","last_active":"2024-04-25T23:52:42.391498+0000","last_peered":"2024-04-25T23:52:42.391498+0000","last_clean":"2024-04-25T23:52:42.391498+0000","last_became_active":"2024-04-25T23:48:58.914379+0000","last_became_peered":"2024-04-25T23:48:58.914379+0000","last_unstale":"2024-04-25T23:52:42.391498+0000","last_undegraded":"2024-04-25T23:52:42.391498+0000","last_fullsized":"2024-04-25T23:52:42.391498+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_clean_scrub_stamp":"2024-04-25T23:48:57.875228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:11:09.852719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":172,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.300390+0000","last_change":"2024-04-25T23:48:59.124853+0000","last_active":"2024-04-25T23:52:42.300390+0000","last_peered":"2024-04-25T23:52:42.300390+0000","last_clean":"2024-04-25T23:52:42.300390+0000","last_became_active":"2024-04-25T23:48:59.122379+0000","last_became_peered":"2024-04-25T23:48:59.122379+0000","last_unstale":"2024-04-25T23:52:42.300390+0000","last_undegraded":"2024-04-25T23:52:42.300390+0000","last_fullsized":"2024-04-25T23:52:42.300390+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_clean_scrub_stamp":"2024-04-25T23:48:57.875228+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:57:31.186906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"21'3897","reported_seq":3938,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.802677+0000","last_change":"2024-04-25T23:49:04.980999+0000","last_active":"2024-04-25T23:52:42.802677+0000","last_peered":"2024-04-25T23:52:42.802677+0000","last_clean":"2024-04-25T23:52:42.802677+0000","last_became_active":"2024-04-25T23:49:04.980272+0000","last_became_peered":"2024-04-25T23:49:04.980272+0000","last_unstale":"2024-04-25T23:52:42.802677+0000","last_undegraded":"2024-04-25T23:52:42.802677+0000","last_fullsized":"2024-04-25T23:52:42.802677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":3897,"log_dups_size":0,"ondisk_log_size":3897,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:43:10.867606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100794368,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":94,"num_read_kb":77,"num_write":3859,"num_write_kb":201602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"21'7829","reported_seq":7966,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:53:57.613005+0000","last_change":"2024-04-25T23:49:05.094703+0000","last_active":"2024-04-25T23:53:57.613005+0000","last_peered":"2024-04-25T23:53:57.613005+0000","last_clean":"2024-04-25T23:53:57.613005+0000","last_became_active":"2024-04-25T23:49:05.094007+0000","last_became_peered":"2024-04-25T23:49:05.094007+0000","last_unstale":"2024-04-25T23:53:57.613005+0000","last_undegraded":"2024-04-25T23:53:57.613005+0000","last_fullsized":"2024-04-25T23:53:57.613005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":7829,"log_dups_size":0,"ondisk_log_size":7829,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:33:21.090339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":95027712,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":414,"num_read_kb":363,"num_write":5838,"num_write_kb":183309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"21'4056","reported_seq":4079,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:53:47.643827+0000","last_change":"2024-04-25T23:49:04.983367+0000","last_active":"2024-04-25T23:53:47.643827+0000","last_peered":"2024-04-25T23:53:47.643827+0000","last_clean":"2024-04-25T23:53:47.643827+0000","last_became_active":"2024-04-25T23:49:04.982660+0000","last_became_peered":"2024-04-25T23:49:04.982660+0000","last_unstale":"2024-04-25T23:53:47.643827+0000","last_undegraded":"2024-04-25T23:53:47.643827+0000","last_fullsized":"2024-04-25T23:53:47.643827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":4056,"log_dups_size":0,"ondisk_log_size":4056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:34:08.771639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123396624,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":13,"num_read_kb":11,"num_write":4049,"num_write_kb":164094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"20'3795","reported_seq":3811,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.302007+0000","last_change":"2024-04-25T23:49:04.991004+0000","last_active":"2024-04-25T23:52:42.302007+0000","last_peered":"2024-04-25T23:52:42.302007+0000","last_clean":"2024-04-25T23:52:42.302007+0000","last_became_active":"2024-04-25T23:49:04.990144+0000","last_became_peered":"2024-04-25T23:49:04.990144+0000","last_unstale":"2024-04-25T23:52:42.302007+0000","last_undegraded":"2024-04-25T23:52:42.302007+0000","last_fullsized":"2024-04-25T23:52:42.302007+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":3795,"log_dups_size":0,"ondisk_log_size":3795,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:18:20.437365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113246208,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3792,"num_write_kb":169093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"21'3586","reported_seq":3613,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.794253+0000","last_change":"2024-04-25T23:49:04.988518+0000","last_active":"2024-04-25T23:52:42.794253+0000","last_peered":"2024-04-25T23:52:42.794253+0000","last_clean":"2024-04-25T23:52:42.794253+0000","last_became_active":"2024-04-25T23:49:04.987663+0000","last_became_peered":"2024-04-25T23:49:04.987663+0000","last_unstale":"2024-04-25T23:52:42.794253+0000","last_undegraded":"2024-04-25T23:52:42.794253+0000","last_fullsized":"2024-04-25T23:52:42.794253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":3586,"log_dups_size":0,"ondisk_log_size":3586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:08:28.379346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112721920,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":45,"num_read_kb":34,"num_write":3560,"num_write_kb":152196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"21'4479","reported_seq":4500,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.800823+0000","last_change":"2024-04-25T23:49:04.997861+0000","last_active":"2024-04-25T23:52:42.800823+0000","last_peered":"2024-04-25T23:52:42.800823+0000","last_clean":"2024-04-25T23:52:42.800823+0000","last_became_active":"2024-04-25T23:49:04.996781+0000","last_became_peered":"2024-04-25T23:49:04.996781+0000","last_unstale":"2024-04-25T23:52:42.800823+0000","last_undegraded":"2024-04-25T23:52:42.800823+0000","last_fullsized":"2024-04-25T23:52:42.800823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":4479,"log_dups_size":0,"ondisk_log_size":4479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:48:47.865840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":127338152,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":8,"num_read_kb":7,"num_write":4466,"num_write_kb":191484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"21'5770","reported_seq":5861,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.888218+0000","last_change":"2024-04-25T23:49:04.998785+0000","last_active":"2024-04-25T23:52:42.888218+0000","last_peered":"2024-04-25T23:52:42.888218+0000","last_clean":"2024-04-25T23:52:42.888218+0000","last_became_active":"2024-04-25T23:49:04.997977+0000","last_became_peered":"2024-04-25T23:49:04.997977+0000","last_unstale":"2024-04-25T23:52:42.888218+0000","last_undegraded":"2024-04-25T23:52:42.888218+0000","last_fullsized":"2024-04-25T23:52:42.888218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":5770,"log_dups_size":0,"ondisk_log_size":5770,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:13:53.271710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":116271984,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":286,"num_read_kb":214,"num_write":6036,"num_write_kb":177360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"21'3327","reported_seq":3343,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.804453+0000","last_change":"2024-04-25T23:49:04.998698+0000","last_active":"2024-04-25T23:52:42.804453+0000","last_peered":"2024-04-25T23:52:42.804453+0000","last_clean":"2024-04-25T23:52:42.804453+0000","last_became_active":"2024-04-25T23:49:04.997986+0000","last_became_peered":"2024-04-25T23:49:04.997986+0000","last_unstale":"2024-04-25T23:52:42.804453+0000","last_undegraded":"2024-04-25T23:52:42.804453+0000","last_fullsized":"2024-04-25T23:52:42.804453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":3327,"log_dups_size":0,"ondisk_log_size":3327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:59:11.386821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663296,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3277,"num_write_kb":152452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"21'3069","reported_seq":3085,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.796063+0000","last_change":"2024-04-25T23:49:04.990446+0000","last_active":"2024-04-25T23:52:42.796063+0000","last_peered":"2024-04-25T23:52:42.796063+0000","last_clean":"2024-04-25T23:52:42.796063+0000","last_became_active":"2024-04-25T23:49:04.989709+0000","last_became_peered":"2024-04-25T23:49:04.989709+0000","last_unstale":"2024-04-25T23:52:42.796063+0000","last_undegraded":"2024-04-25T23:52:42.796063+0000","last_fullsized":"2024-04-25T23:52:42.796063+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":3069,"log_dups_size":0,"ondisk_log_size":3069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:22:01.360709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3021,"num_write_kb":144132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"20'4577","reported_seq":4601,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.299824+0000","last_change":"2024-04-25T23:49:04.998536+0000","last_active":"2024-04-25T23:52:42.299824+0000","last_peered":"2024-04-25T23:52:42.299824+0000","last_clean":"2024-04-25T23:52:42.299824+0000","last_became_active":"2024-04-25T23:49:04.997718+0000","last_became_peered":"2024-04-25T23:49:04.997718+0000","last_unstale":"2024-04-25T23:52:42.299824+0000","last_undegraded":"2024-04-25T23:52:42.299824+0000","last_fullsized":"2024-04-25T23:52:42.299824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":4577,"log_dups_size":0,"ondisk_log_size":4577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:42:48.145557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117440512,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":4584,"num_write_kb":172941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"21'4926","reported_seq":4942,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:42.806464+0000","last_change":"2024-04-25T23:49:04.988912+0000","last_active":"2024-04-25T23:52:42.806464+0000","last_peered":"2024-04-25T23:52:42.806464+0000","last_clean":"2024-04-25T23:52:42.806464+0000","last_became_active":"2024-04-25T23:49:04.987367+0000","last_became_peered":"2024-04-25T23:49:04.987367+0000","last_unstale":"2024-04-25T23:52:42.806464+0000","last_undegraded":"2024-04-25T23:52:42.806464+0000","last_fullsized":"2024-04-25T23:52:42.806464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":4926,"log_dups_size":0,"ondisk_log_size":4926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:11:40.900415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":149553152,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4881,"num_write_kb":166152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"21'7343","reported_seq":7369,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T23:52:55.941996+0000","last_change":"2024-04-25T23:49:04.997739+0000","last_active":"2024-04-25T23:52:55.941996+0000","last_peered":"2024-04-25T23:52:55.941996+0000","last_clean":"2024-04-25T23:52:55.941996+0000","last_became_active":"2024-04-25T23:49:04.996841+0000","last_became_peered":"2024-04-25T23:49:04.996841+0000","last_unstale":"2024-04-25T23:52:55.941996+0000","last_undegraded":"2024-04-25T23:52:55.941996+0000","last_fullsized":"2024-04-25T23:52:55.941996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":7343,"log_dups_size":0,"ondisk_log_size":7343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:08:44.872965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83346448,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":33,"num_read_kb":88,"num_write":7349,"num_write_kb":122645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":3560846088,"num_objects":877,"num_object_clones":0,"num_object_copies":1754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":877,"num_whiteouts":0,"num_read":2507,"num_read_kb":2518,"num_write":155692,"num_write_kb":5620473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":6782795776,"data_stored":6820341912,"data_compressed":194288578,"data_compressed_allocated":202072064,"data_compressed_original":243892224,"omap_allocated":0,"internal_metadata":0},"log_size":157484,"ondisk_log_size":157484,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":12118,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542206,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":388020,"kb_used_data":298604,"kb_used_omap":1,"kb_used_meta":89406,"kb_avail":93983820,"statfs":{"total":96636764160,"available":96239431680,"internally_reserved":0,"allocated":305770496,"data_stored":306544866,"data_compressed":6553827,"data_compressed_allocated":6823936,"data_compressed_original":7905280,"omap_allocated":1593,"internal_metadata":91552199},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542208,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1212816,"kb_used_data":929032,"kb_used_omap":1,"kb_used_meta":283774,"kb_avail":93159024,"statfs":{"total":96636764160,"available":95394840576,"internally_reserved":0,"allocated":951328768,"data_stored":954362994,"data_compressed":23747576,"data_compressed_allocated":24735744,"data_compressed_original":28565504,"omap_allocated":1595,"internal_metadata":290585029},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542206,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1859000,"kb_used_data":1332336,"kb_used_omap":1,"kb_used_meta":526654,"kb_avail":92512840,"statfs":{"total":96636764160,"available":94733148160,"internally_reserved":0,"allocated":1364312064,"data_stored":1378232810,"data_compressed":53769970,"data_compressed_allocated":55988224,"data_compressed_original":70811648,"omap_allocated":1595,"internal_metadata":539294149},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574910,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1460108,"kb_used_data":1031236,"kb_used_omap":1,"kb_used_meta":428862,"kb_avail":92911732,"statfs":{"total":96636764160,"available":95141613568,"internally_reserved":0,"allocated":1055985664,"data_stored":1063157334,"data_compressed":33533197,"data_compressed_allocated":34906112,"data_compressed_original":43012096,"omap_allocated":1590,"internal_metadata":439155146},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574911,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1418012,"kb_used_data":1062036,"kb_used_omap":1,"kb_used_meta":355966,"kb_avail":92953828,"statfs":{"total":96636764160,"available":95184719872,"internally_reserved":0,"allocated":1087524864,"data_stored":1089954434,"data_compressed":21531874,"data_compressed_allocated":22360064,"data_compressed_original":25559040,"omap_allocated":1593,"internal_metadata":364509639},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574910,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":424748,"kb_used_data":303332,"kb_used_omap":1,"kb_used_meta":121406,"kb_avail":93947092,"statfs":{"total":96636764160,"available":96201822208,"internally_reserved":0,"allocated":310611968,"data_stored":314944206,"data_compressed":11229575,"data_compressed_allocated":11784192,"data_compressed_original":16420864,"omap_allocated":1595,"internal_metadata":124320197},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":861340,"kb_used_data":623380,"kb_used_omap":1,"kb_used_meta":237950,"kb_avail":93510500,"statfs":{"total":96636764160,"available":95754752000,"internally_reserved":0,"allocated":638341120,"data_stored":641442682,"data_compressed":20262986,"data_compressed_allocated":21012480,"data_compressed_original":24612864,"omap_allocated":1595,"internal_metadata":243661253},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574910,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1391756,"kb_used_data":1045508,"kb_used_omap":1,"kb_used_meta":346238,"kb_avail":92980084,"statfs":{"total":96636764160,"available":95211606016,"internally_reserved":0,"allocated":1070600192,"data_stored":1073278922,"data_compressed":23671691,"data_compressed_allocated":24526848,"data_compressed_original":28028928,"omap_allocated":1595,"internal_metadata":354548165},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":6059,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":6059,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1070350336,"data_stored":1072685184,"data_compressed":23665632,"data_compressed_allocated":24494080,"data_compressed_original":27516928,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":638144512,"data_stored":641377872,"data_compressed":20262986,"data_compressed_allocated":21012480,"data_compressed_original":24612864,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":310415360,"data_stored":314879396,"data_compressed":11229575,"data_compressed_allocated":11784192,"data_compressed_original":16420864,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1087328256,"data_stored":1089889624,"data_compressed":21531874,"data_compressed_allocated":22360064,"data_compressed_original":25559040,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1364115456,"data_stored":1378168000,"data_compressed":53769970,"data_compressed_allocated":55988224,"data_compressed_original":70811648,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":951132160,"data_stored":954298184,"data_compressed":23747576,"data_compressed_allocated":24735744,"data_compressed_original":28565504,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":305573888,"data_stored":306480056,"data_compressed":6553827,"data_compressed_allocated":6823936,"data_compressed_original":7905280,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1055735808,"data_stored":1062563596,"data_compressed":33527138,"data_compressed_allocated":34873344,"data_compressed_original":42500096,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T23:53:59.851 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T23:53:59.972 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.988+0000 7f1eb0f55640 1 Processor -- start 2024-04-25T23:53:59.973 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.988+0000 7f1eb0f55640 1 -- start start 2024-04-25T23:53:59.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.988+0000 7f1eb0f55640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1eac06bac0 0x7f1eac143150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:53:59.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.988+0000 7f1eb0f55640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f1eac06f410 0x7f1eac142540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:53:59.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.988+0000 7f1eb0f55640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1eac06bea0 con 0x7f1eac071fc0 2024-04-25T23:53:59.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.988+0000 7f1eb0f55640 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 -- 0x7f1eac142cc0 con 0x7f1eac06f410 2024-04-25T23:53:59.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.988+0000 7f1eb0f55640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f1eac072430 con 0x7f1eac06bac0 2024-04-25T23:53:59.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.988+0000 7f1eaad9d640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f1eac06f410 0x7f1eac142540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:53:59.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.988+0000 7f1eaad9d640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f1eac06f410 0x7f1eac142540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37138/0 (socket says 172.21.15.57:37138) 2024-04-25T23:53:59.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.988+0000 7f1eaad9d640 1 -- 172.21.15.57:0/4228859140 learned_addr learned my addr 172.21.15.57:0/4228859140 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:53:59.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.988+0000 7f1eaa59c640 1 --2- 172.21.15.57:0/4228859140 >> v2:172.21.15.80:3300/0 conn(0x7f1eac06bac0 0x7f1eac143150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:53:59.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.988+0000 7f1eaad9d640 1 -- 172.21.15.57:0/4228859140 >> v2:172.21.15.80:3300/0 conn(0x7f1eac06bac0 msgr2=0x7f1eac143150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:53:59.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.988+0000 7f1eaad9d640 1 --2- 172.21.15.57:0/4228859140 >> v2:172.21.15.80:3300/0 conn(0x7f1eac06bac0 0x7f1eac143150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:53:59.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.988+0000 7f1eaad9d640 1 -- 172.21.15.57:0/4228859140 >> v1:172.21.15.57:6789/0 conn(0x7f1eac071fc0 legacy=0x7f1eac141e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:53:59.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.988+0000 7f1eaad9d640 1 -- 172.21.15.57:0/4228859140 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eac143690 con 0x7f1eac06f410 2024-04-25T23:53:59.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.992+0000 7f1eaad9d640 1 --2- 172.21.15.57:0/4228859140 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1eac06f410 0x7f1eac142540 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1eac0b5c50 tx=0x7f1ea401c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e1420675e67ab1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:53:59.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.992+0000 7f1ea959a640 1 -- 172.21.15.57:0/4228859140 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea40a5690 con 0x7f1eac06f410 2024-04-25T23:53:59.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.992+0000 7f1ea959a640 1 -- 172.21.15.57:0/4228859140 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ea40a5830 con 0x7f1eac06f410 2024-04-25T23:53:59.976 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.992+0000 7f1ea959a640 1 -- 172.21.15.57:0/4228859140 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea40a5c00 con 0x7f1eac06f410 2024-04-25T23:53:59.977 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.992+0000 7f1eb0f55640 1 -- 172.21.15.57:0/4228859140 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1eac06f410 msgr2=0x7f1eac142540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:53:59.977 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.992+0000 7f1eb0f55640 1 --2- 172.21.15.57:0/4228859140 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1eac06f410 0x7f1eac142540 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1eac0b5c50 tx=0x7f1ea401c940 comp rx=0 tx=0).stop 2024-04-25T23:53:59.977 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.992+0000 7f1eb0f55640 1 -- 172.21.15.57:0/4228859140 shutdown_connections 2024-04-25T23:53:59.977 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.992+0000 7f1eb0f55640 1 --2- 172.21.15.57:0/4228859140 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1eac06f410 0x7f1eac142540 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:53:59.977 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.992+0000 7f1eb0f55640 1 --2- 172.21.15.57:0/4228859140 >> v2:172.21.15.80:3300/0 conn(0x7f1eac06bac0 0x7f1eac143150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:53:59.977 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.992+0000 7f1eb0f55640 1 -- 172.21.15.57:0/4228859140 >> 172.21.15.57:0/4228859140 conn(0x7f1eac06cfd0 msgr2=0x7f1eac0b5090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:53:59.977 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.992+0000 7f1eb0f55640 1 -- 172.21.15.57:0/4228859140 shutdown_connections 2024-04-25T23:53:59.977 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.992+0000 7f1eb0f55640 1 -- 172.21.15.57:0/4228859140 wait complete. 2024-04-25T23:53:59.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.992+0000 7f1eb0f55640 1 Processor -- start 2024-04-25T23:53:59.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.992+0000 7f1eb0f55640 1 -- start start 2024-04-25T23:53:59.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.992+0000 7f1eb0f55640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1eac06bac0 0x7f1eac13de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:53:59.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.992+0000 7f1eb0f55640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1eac06f410 0x7f1eac0b07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:53:59.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.992+0000 7f1eb0f55640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f1eac071fc0 0x7f1eac0aff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:53:59.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.992+0000 7f1eb0f55640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f1eac14fe40 con 0x7f1eac06f410 2024-04-25T23:53:59.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.992+0000 7f1eb0f55640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f1eac0725b0 con 0x7f1eac06bac0 2024-04-25T23:53:59.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.992+0000 7f1eb0f55640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f1eac14ffc0 con 0x7f1eac071fc0 2024-04-25T23:53:59.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.992+0000 7f1ea9d9b640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1eac06f410 0x7f1eac0b07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:53:59.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.992+0000 7f1eaad9d640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f1eac071fc0 0x7f1eac0aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:53:59.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.992+0000 7f1ea9d9b640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1eac06f410 0x7f1eac0b07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37154/0 (socket says 172.21.15.57:37154) 2024-04-25T23:53:59.980 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.992+0000 7f1ea9d9b640 1 -- 172.21.15.57:0/3654773343 learned_addr learned my addr 172.21.15.57:0/3654773343 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:53:59.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.992+0000 7f1eaad9d640 1 -- 172.21.15.57:0/3654773343 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f1eac071fc0 msgr2=0x7f1eac0aff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-04-25T23:53:59.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.992+0000 7f1eaad9d640 1 -- 172.21.15.57:0/3654773343 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f1eac071fc0 msgr2=0x7f1eac0aff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T23:53:59.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.992+0000 7f1eaad9d640 1 --2- 172.21.15.57:0/3654773343 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f1eac071fc0 0x7f1eac0aff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T23:53:59.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.992+0000 7f1eaad9d640 1 --2- 172.21.15.57:0/3654773343 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f1eac071fc0 0x7f1eac0aff20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T23:53:59.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.992+0000 7f1eaa59c640 1 --2- 172.21.15.57:0/3654773343 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1eac06bac0 0x7f1eac13de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:53:59.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.992+0000 7f1ea9d9b640 1 -- 172.21.15.57:0/3654773343 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f1eac071fc0 msgr2=0x7f1eac0aff20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T23:53:59.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.992+0000 7f1ea9d9b640 1 --2- 172.21.15.57:0/3654773343 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f1eac071fc0 0x7f1eac0aff20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:53:59.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.992+0000 7f1ea9d9b640 1 -- 172.21.15.57:0/3654773343 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1eac06bac0 msgr2=0x7f1eac13de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:53:59.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.992+0000 7f1ea9d9b640 1 --2- 172.21.15.57:0/3654773343 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1eac06bac0 0x7f1eac13de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:53:59.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.992+0000 7f1ea9d9b640 1 -- 172.21.15.57:0/3654773343 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eac0afa00 con 0x7f1eac06f410 2024-04-25T23:53:59.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.996+0000 7f1ea9d9b640 1 --2- 172.21.15.57:0/3654773343 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1eac06f410 0x7f1eac0b07f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1ea000c3a0 tx=0x7f1ea000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:53:59.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.996+0000 7f1e9affd640 1 -- 172.21.15.57:0/3654773343 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea00052d0 con 0x7f1eac06f410 2024-04-25T23:53:59.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.996+0000 7f1e9affd640 1 -- 172.21.15.57:0/3654773343 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ea0005470 con 0x7f1eac06f410 2024-04-25T23:53:59.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.996+0000 7f1eb0f55640 1 -- 172.21.15.57:0/3654773343 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1eac137c70 con 0x7f1eac06f410 2024-04-25T23:53:59.981 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.996+0000 7f1e9affd640 1 -- 172.21.15.57:0/3654773343 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea0007260 con 0x7f1eac06f410 2024-04-25T23:53:59.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.996+0000 7f1eb0f55640 1 -- 172.21.15.57:0/3654773343 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1eac1de5e0 con 0x7f1eac06f410 2024-04-25T23:53:59.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.996+0000 7f1eb0f55640 1 -- 172.21.15.57:0/3654773343 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1eac0761e0 con 0x7f1eac06f410 2024-04-25T23:53:59.983 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.996+0000 7f1e9affd640 1 -- 172.21.15.57:0/3654773343 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) ==== 87410+0+0 (secure 0 0 0) 0x7f1ea0007400 con 0x7f1eac06f410 2024-04-25T23:53:59.983 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.996+0000 7f1e9affd640 1 --2- 172.21.15.57:0/3654773343 >> v2:172.21.15.80:6816/2493457677 conn(0x7f1e8c067880 0x7f1e8c069d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:53:59.983 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.996+0000 7f1e9affd640 1 -- 172.21.15.57:0/3654773343 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 4525+0+0 (secure 0 0 0) 0x7f1ea0085450 con 0x7f1eac06f410 2024-04-25T23:53:59.983 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.996+0000 7f1eaa59c640 1 --2- 172.21.15.57:0/3654773343 >> v2:172.21.15.80:6816/2493457677 conn(0x7f1e8c067880 0x7f1e8c069d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:53:59.983 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:53:59.996+0000 7f1eaa59c640 1 --2- 172.21.15.57:0/3654773343 >> v2:172.21.15.80:6816/2493457677 conn(0x7f1e8c067880 0x7f1e8c069d40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c00b9d0 tx=0x7f1e9c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:53:59.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:00.000+0000 7f1e9affd640 1 -- 172.21.15.57:0/3654773343 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1eac06bea0 con 0x7f1eac06f410 2024-04-25T23:54:00.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:00.164+0000 7f1eb0f55640 1 -- 172.21.15.57:0/3654773343 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1eac0afbd0 con 0x7f1eac06f410 2024-04-25T23:54:00.149 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:00.164+0000 7f1e9affd640 1 -- 172.21.15.57:0/3654773343 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+14633 (secure 0 0 0) 0x7f1ea00564a0 con 0x7f1eac06f410 2024-04-25T23:54:00.150 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-25T23:54:00.150 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":22,"fsid":"ca0adda3-98dd-4ba5-b4b2-8b4f20dcabdd","created":"2024-04-25T23:48:46.211894+0000","modified":"2024-04-25T23:53:58.185544+0000","last_up_change":"2024-04-25T23:48:56.859933+0000","last_in_change":"2024-04-25T23:48:48.838840+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T23:48:56.919796+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-25T23:48:57.576441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephsqlite","create_time":"2024-04-25T23:49:03.821982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5986c317-8828-4568-af68-a8ae96690c51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":2528545516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6813","nonce":2528545516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6815","nonce":2528545516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":2528545516}]},"public_addr":"172.21.15.57:6812/2528545516","cluster_addr":"172.21.15.57:6813/2528545516","heartbeat_back_addr":"172.21.15.57:6815/2528545516","heartbeat_front_addr":"172.21.15.57:6814/2528545516","state":["exists","up"]},{"osd":1,"uuid":"1aa157dd-52fe-4bb3-9f63-4db493de5333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":2225240106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6809","nonce":2225240106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6811","nonce":2225240106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":2225240106}]},"public_addr":"172.21.15.57:6808/2225240106","cluster_addr":"172.21.15.57:6809/2225240106","heartbeat_back_addr":"172.21.15.57:6811/2225240106","heartbeat_front_addr":"172.21.15.57:6810/2225240106","state":["exists","up"]},{"osd":2,"uuid":"1391be38-25ac-43f0-8090-15adddc6d69d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":1976583347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6805","nonce":1976583347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6807","nonce":1976583347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":1976583347}]},"public_addr":"172.21.15.57:6804/1976583347","cluster_addr":"172.21.15.57:6805/1976583347","heartbeat_back_addr":"172.21.15.57:6807/1976583347","heartbeat_front_addr":"172.21.15.57:6806/1976583347","state":["exists","up"]},{"osd":3,"uuid":"a85e7650-effb-4ea7-9f3b-9fc9fc1576e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":1899803014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6801","nonce":1899803014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6803","nonce":1899803014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":1899803014}]},"public_addr":"172.21.15.57:6800/1899803014","cluster_addr":"172.21.15.57:6801/1899803014","heartbeat_back_addr":"172.21.15.57:6803/1899803014","heartbeat_front_addr":"172.21.15.57:6802/1899803014","state":["exists","up"]},{"osd":4,"uuid":"ad3fb1a9-cec6-4fb3-9f49-22b9480f524b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":4069821322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":4069821322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":4069821322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":4069821322}]},"public_addr":"172.21.15.80:6800/4069821322","cluster_addr":"172.21.15.80:6802/4069821322","heartbeat_back_addr":"172.21.15.80:6806/4069821322","heartbeat_front_addr":"172.21.15.80:6804/4069821322","state":["exists","up"]},{"osd":5,"uuid":"fd05cda6-27a5-477a-8916-b34b69ba9ef6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6811","nonce":2954555099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":2954555099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6815","nonce":2954555099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2954555099}]},"public_addr":"172.21.15.80:6811/2954555099","cluster_addr":"172.21.15.80:6813/2954555099","heartbeat_back_addr":"172.21.15.80:6815/2954555099","heartbeat_front_addr":"172.21.15.80:6814/2954555099","state":["exists","up"]},{"osd":6,"uuid":"599ef421-483d-4225-a98f-7405006b1809","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6801","nonce":163896711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6803","nonce":163896711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6807","nonce":163896711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6805","nonce":163896711}]},"public_addr":"172.21.15.80:6801/163896711","cluster_addr":"172.21.15.80:6803/163896711","heartbeat_back_addr":"172.21.15.80:6807/163896711","heartbeat_front_addr":"172.21.15.80:6805/163896711","state":["exists","up"]},{"osd":7,"uuid":"0e9d533c-465f-403c-8fe8-7dafc953d774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2647667231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":2647667231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2647667231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2647667231}]},"public_addr":"172.21.15.80:6808/2647667231","cluster_addr":"172.21.15.80:6809/2647667231","heartbeat_back_addr":"172.21.15.80:6812/2647667231","heartbeat_front_addr":"172.21.15.80:6810/2647667231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T23:48:54.460484+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T23:48:54.420671+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T23:48:54.335197+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T23:48:54.243121+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T23:48:54.249670+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T23:48:54.153679+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T23:48:53.992685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T23:48:54.129310+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/4286684903":"2024-04-26T00:53:57.615039+0000","172.21.15.57:0/803088574":"2024-04-26T00:52:41.692623+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T23:54:00.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:00.168+0000 7f1eb0f55640 1 -- 172.21.15.57:0/3654773343 >> v2:172.21.15.80:6816/2493457677 conn(0x7f1e8c067880 msgr2=0x7f1e8c069d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:00.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:00.168+0000 7f1eb0f55640 1 --2- 172.21.15.57:0/3654773343 >> v2:172.21.15.80:6816/2493457677 conn(0x7f1e8c067880 0x7f1e8c069d40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c00b9d0 tx=0x7f1e9c008040 comp rx=0 tx=0).stop 2024-04-25T23:54:00.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:00.168+0000 7f1eb0f55640 1 -- 172.21.15.57:0/3654773343 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1eac06f410 msgr2=0x7f1eac0b07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:00.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:00.168+0000 7f1eb0f55640 1 --2- 172.21.15.57:0/3654773343 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1eac06f410 0x7f1eac0b07f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1ea000c3a0 tx=0x7f1ea000c860 comp rx=0 tx=0).stop 2024-04-25T23:54:00.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:00.168+0000 7f1eb0f55640 1 -- 172.21.15.57:0/3654773343 shutdown_connections 2024-04-25T23:54:00.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:00.168+0000 7f1eb0f55640 1 --2- 172.21.15.57:0/3654773343 >> v2:172.21.15.80:6816/2493457677 conn(0x7f1e8c067880 0x7f1e8c069d40 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:00.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:00.168+0000 7f1eb0f55640 1 --2- 172.21.15.57:0/3654773343 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f1eac071fc0 0x7f1eac0aff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:00.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:00.168+0000 7f1eb0f55640 1 --2- 172.21.15.57:0/3654773343 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1eac06f410 0x7f1eac0b07f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:00.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:00.168+0000 7f1eb0f55640 1 --2- 172.21.15.57:0/3654773343 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1eac06bac0 0x7f1eac13de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:00.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:00.168+0000 7f1eb0f55640 1 -- 172.21.15.57:0/3654773343 >> 172.21.15.57:0/3654773343 conn(0x7f1eac06cfd0 msgr2=0x7f1eac13f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:54:00.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:00.168+0000 7f1eb0f55640 1 -- 172.21.15.57:0/3654773343 shutdown_connections 2024-04-25T23:54:00.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:00.168+0000 7f1eb0f55640 1 -- 172.21.15.57:0/3654773343 wait complete. 2024-04-25T23:54:01.171 INFO:tasks.ceph:Scrubbing osd.0 2024-04-25T23:54:01.171 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-25T23:54:01.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.328+0000 7fb706c35640 1 Processor -- start 2024-04-25T23:54:01.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.328+0000 7fb706c35640 1 -- start start 2024-04-25T23:54:01.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.328+0000 7fb706c35640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fb700071f10 0x7fb700072bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:01.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.328+0000 7fb706c35640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fb7000b42e0 0x7fb700146620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:01.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.328+0000 7fb706c35640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7000722f0 con 0x7fb700146b60 2024-04-25T23:54:01.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.328+0000 7fb706c35640 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 -- 0x7fb700146190 con 0x7fb7000b42e0 2024-04-25T23:54:01.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.328+0000 7fb706c35640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7fb700146ff0 con 0x7fb700071f10 2024-04-25T23:54:01.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.332+0000 7fb6fffff640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fb7000b42e0 0x7fb700146620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:01.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.332+0000 7fb7049d1640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fb700071f10 0x7fb700072bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:01.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.332+0000 7fb6fffff640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fb7000b42e0 0x7fb700146620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37164/0 (socket says 172.21.15.57:37164) 2024-04-25T23:54:01.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.332+0000 7fb6fffff640 1 -- 172.21.15.57:0/2195325305 learned_addr learned my addr 172.21.15.57:0/2195325305 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:54:01.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.332+0000 7fb7049d1640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fb700071f10 0x7fb700072bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3301/0 says I am v2:172.21.15.57:48640/0 (socket says 172.21.15.57:48640) 2024-04-25T23:54:01.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.332+0000 7fb6fffff640 1 -- 172.21.15.57:0/2195325305 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fb700071f10 msgr2=0x7fb700072bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:54:01.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.332+0000 7fb6fffff640 1 --2- 172.21.15.57:0/2195325305 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fb700071f10 0x7fb700072bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:01.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.332+0000 7fb6fffff640 1 -- 172.21.15.57:0/2195325305 >> v1:172.21.15.80:6789/0 conn(0x7fb700146b60 legacy=0x7fb700141fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:54:01.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.332+0000 7fb6fffff640 1 -- 172.21.15.57:0/2195325305 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb70006fca0 con 0x7fb7000b42e0 2024-04-25T23:54:01.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.332+0000 7fb6fffff640 1 --2- 172.21.15.57:0/2195325305 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7000b42e0 0x7fb700146620 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec009870 tx=0x7fb6ec01c290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52fc629123660ca2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:54:01.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.332+0000 7fb6ff7fe640 1 -- 172.21.15.57:0/2195325305 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec0b0070 con 0x7fb7000b42e0 2024-04-25T23:54:01.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.332+0000 7fb6ff7fe640 1 -- 172.21.15.57:0/2195325305 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6ec0ab600 con 0x7fb7000b42e0 2024-04-25T23:54:01.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.332+0000 7fb6ff7fe640 1 -- 172.21.15.57:0/2195325305 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb700142e60 con 0x7fb7000b42e0 2024-04-25T23:54:01.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.332+0000 7fb706c35640 1 -- 172.21.15.57:0/2195325305 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7000b42e0 msgr2=0x7fb700146620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:01.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.332+0000 7fb706c35640 1 --2- 172.21.15.57:0/2195325305 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7000b42e0 0x7fb700146620 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec009870 tx=0x7fb6ec01c290 comp rx=0 tx=0).stop 2024-04-25T23:54:01.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.332+0000 7fb706c35640 1 -- 172.21.15.57:0/2195325305 shutdown_connections 2024-04-25T23:54:01.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.332+0000 7fb706c35640 1 --2- 172.21.15.57:0/2195325305 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7000b42e0 0x7fb700146620 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:01.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.332+0000 7fb706c35640 1 --2- 172.21.15.57:0/2195325305 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fb700071f10 0x7fb700072bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:01.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.332+0000 7fb706c35640 1 -- 172.21.15.57:0/2195325305 >> 172.21.15.57:0/2195325305 conn(0x7fb7000b3490 msgr2=0x7fb7000b38a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:54:01.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.332+0000 7fb706c35640 1 -- 172.21.15.57:0/2195325305 shutdown_connections 2024-04-25T23:54:01.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.332+0000 7fb706c35640 1 -- 172.21.15.57:0/2195325305 wait complete. 2024-04-25T23:54:01.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.336+0000 7fb706c35640 1 Processor -- start 2024-04-25T23:54:01.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.336+0000 7fb706c35640 1 -- start start 2024-04-25T23:54:01.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.336+0000 7fb706c35640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb700071f10 0x7fb70013a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:01.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.336+0000 7fb706c35640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb7000b42e0 0x7fb70013e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:01.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.336+0000 7fb706c35640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fb700146b60 0x7fb7001de800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:01.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.336+0000 7fb706c35640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fb7001500d0 con 0x7fb700071f10 2024-04-25T23:54:01.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.336+0000 7fb706c35640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb70014ff50 con 0x7fb7000b42e0 2024-04-25T23:54:01.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.336+0000 7fb706c35640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7fb700150250 con 0x7fb700146b60 2024-04-25T23:54:01.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.336+0000 7fb7049d1640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb700071f10 0x7fb70013a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:01.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.336+0000 7fb7051d2640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fb700146b60 0x7fb7001de800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:01.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.336+0000 7fb7049d1640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb700071f10 0x7fb70013a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37178/0 (socket says 172.21.15.57:37178) 2024-04-25T23:54:01.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.336+0000 7fb7049d1640 1 -- 172.21.15.57:0/4023960564 learned_addr learned my addr 172.21.15.57:0/4023960564 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:54:01.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.336+0000 7fb7051d2640 1 --2- 172.21.15.57:0/4023960564 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fb700146b60 0x7fb7001de800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3301/0 says I am v2:172.21.15.57:48650/0 (socket says 172.21.15.57:48650) 2024-04-25T23:54:01.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.336+0000 7fb6fffff640 1 --2- 172.21.15.57:0/4023960564 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb7000b42e0 0x7fb70013e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:01.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.336+0000 7fb7051d2640 1 -- 172.21.15.57:0/4023960564 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb7000b42e0 msgr2=0x7fb70013e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:01.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.336+0000 7fb7051d2640 1 --2- 172.21.15.57:0/4023960564 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb7000b42e0 0x7fb70013e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:01.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.336+0000 7fb7051d2640 1 -- 172.21.15.57:0/4023960564 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb700071f10 msgr2=0x7fb70013a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:01.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.336+0000 7fb7051d2640 1 --2- 172.21.15.57:0/4023960564 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb700071f10 0x7fb70013a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:01.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.336+0000 7fb7051d2640 1 -- 172.21.15.57:0/4023960564 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb70013e9c0 con 0x7fb700146b60 2024-04-25T23:54:01.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.336+0000 7fb7049d1640 1 --2- 172.21.15.57:0/4023960564 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb700071f10 0x7fb70013a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T23:54:01.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.336+0000 7fb7051d2640 1 --2- 172.21.15.57:0/4023960564 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fb700146b60 0x7fb7001de800 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb6f800be00 tx=0x7fb6f800bef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:54:01.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.336+0000 7fb6fd7fa640 1 -- 172.21.15.57:0/4023960564 <== mon.2 v2:172.21.15.57:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f8003980 con 0x7fb700146b60 2024-04-25T23:54:01.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.336+0000 7fb6fd7fa640 1 -- 172.21.15.57:0/4023960564 <== mon.2 v2:172.21.15.57:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6f8003b20 con 0x7fb700146b60 2024-04-25T23:54:01.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.336+0000 7fb6fd7fa640 1 -- 172.21.15.57:0/4023960564 <== mon.2 v2:172.21.15.57:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f8004020 con 0x7fb700146b60 2024-04-25T23:54:01.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.336+0000 7fb706c35640 1 -- 172.21.15.57:0/4023960564 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7001deda0 con 0x7fb700146b60 2024-04-25T23:54:01.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.336+0000 7fb706c35640 1 -- 172.21.15.57:0/4023960564 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb700137190 con 0x7fb700146b60 2024-04-25T23:54:01.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.336+0000 7fb706c35640 1 -- 172.21.15.57:0/4023960564 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb7000761e0 con 0x7fb700146b60 2024-04-25T23:54:01.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.336+0000 7fb6fd7fa640 1 -- 172.21.15.57:0/4023960564 <== mon.2 v2:172.21.15.57:3301/0 4 ==== mgrmap(e 5) ==== 87410+0+0 (secure 0 0 0) 0x7fb6f8006d10 con 0x7fb700146b60 2024-04-25T23:54:01.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.336+0000 7fb6fd7fa640 1 --2- 172.21.15.57:0/4023960564 >> v2:172.21.15.80:6816/2493457677 conn(0x7fb6e4067880 0x7fb6e4069d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:01.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.340+0000 7fb6fd7fa640 1 -- 172.21.15.57:0/4023960564 <== mon.2 v2:172.21.15.57:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 4525+0+0 (secure 0 0 0) 0x7fb6f8086060 con 0x7fb700146b60 2024-04-25T23:54:01.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.340+0000 7fb6fd7fa640 1 --2- 172.21.15.57:0/4023960564 >> v2:172.21.15.57:6812/2528545516 conn(0x7fb6e406e5e0 0x7fb6e4070a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:01.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.340+0000 7fb6fd7fa640 1 -- 172.21.15.57:0/4023960564 --> v2:172.21.15.57:6812/2528545516 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb6e4071070 con 0x7fb6e406e5e0 2024-04-25T23:54:01.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.340+0000 7fb6fd7fa640 1 -- 172.21.15.57:0/4023960564 <== mon.2 v2:172.21.15.57:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fb6f80864f0 con 0x7fb700146b60 2024-04-25T23:54:01.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.340+0000 7fb7049d1640 1 --2- 172.21.15.57:0/4023960564 >> v2:172.21.15.80:6816/2493457677 conn(0x7fb6e4067880 0x7fb6e4069d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:01.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.340+0000 7fb6fffff640 1 --2- 172.21.15.57:0/4023960564 >> v2:172.21.15.57:6812/2528545516 conn(0x7fb6e406e5e0 0x7fb6e4070a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:01.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.340+0000 7fb6fffff640 1 --2- 172.21.15.57:0/4023960564 >> v2:172.21.15.57:6812/2528545516 conn(0x7fb6e406e5e0 0x7fb6e4070a00 crc :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:54:01.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.340+0000 7fb7049d1640 1 --2- 172.21.15.57:0/4023960564 >> v2:172.21.15.80:6816/2493457677 conn(0x7fb6e4067880 0x7fb6e4069d40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb6f4011310 tx=0x7fb6f4009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:54:01.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.340+0000 7fb6fd7fa640 1 -- 172.21.15.57:0/4023960564 <== osd.0 v2:172.21.15.57:6812/2528545516 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb6e4071070 con 0x7fb6e406e5e0 2024-04-25T23:54:01.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.352+0000 7fb706c35640 1 -- 172.21.15.57:0/4023960564 --> v2:172.21.15.57:6812/2528545516 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fb7000b6010 con 0x7fb6e406e5e0 2024-04-25T23:54:01.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.352+0000 7fb6fd7fa640 1 -- 172.21.15.57:0/4023960564 <== osd.0 v2:172.21.15.57:6812/2528545516 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fb7000b6010 con 0x7fb6e406e5e0 2024-04-25T23:54:01.338 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-25T23:54:01.338 INFO:teuthology.orchestra.run.smithi057.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T23:54:01.338 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-25T23:54:01.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.352+0000 7fb706c35640 1 -- 172.21.15.57:0/4023960564 >> v2:172.21.15.57:6812/2528545516 conn(0x7fb6e406e5e0 msgr2=0x7fb6e4070a00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:01.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.352+0000 7fb706c35640 1 --2- 172.21.15.57:0/4023960564 >> v2:172.21.15.57:6812/2528545516 conn(0x7fb6e406e5e0 0x7fb6e4070a00 crc :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:01.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.352+0000 7fb706c35640 1 -- 172.21.15.57:0/4023960564 >> v2:172.21.15.80:6816/2493457677 conn(0x7fb6e4067880 msgr2=0x7fb6e4069d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:01.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.352+0000 7fb706c35640 1 --2- 172.21.15.57:0/4023960564 >> v2:172.21.15.80:6816/2493457677 conn(0x7fb6e4067880 0x7fb6e4069d40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb6f4011310 tx=0x7fb6f4009190 comp rx=0 tx=0).stop 2024-04-25T23:54:01.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.352+0000 7fb706c35640 1 -- 172.21.15.57:0/4023960564 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fb700146b60 msgr2=0x7fb7001de800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:01.340 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.352+0000 7fb706c35640 1 --2- 172.21.15.57:0/4023960564 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fb700146b60 0x7fb7001de800 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb6f800be00 tx=0x7fb6f800bef0 comp rx=0 tx=0).stop 2024-04-25T23:54:01.340 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.352+0000 7fb7051d2640 1 -- 172.21.15.57:0/4023960564 reap_dead start 2024-04-25T23:54:01.340 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.352+0000 7fb706c35640 1 -- 172.21.15.57:0/4023960564 shutdown_connections 2024-04-25T23:54:01.340 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.352+0000 7fb706c35640 1 -- 172.21.15.57:0/4023960564 >> 172.21.15.57:0/4023960564 conn(0x7fb7000b3490 msgr2=0x7fb700066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:54:01.340 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.352+0000 7fb706c35640 1 -- 172.21.15.57:0/4023960564 shutdown_connections 2024-04-25T23:54:01.340 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.352+0000 7fb706c35640 1 -- 172.21.15.57:0/4023960564 wait complete. 2024-04-25T23:54:01.355 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-25T23:54:01.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.488+0000 7fbc226ae640 1 Processor -- start 2024-04-25T23:54:01.472 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.488+0000 7fbc226ae640 1 -- start start 2024-04-25T23:54:01.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.488+0000 7fbc226ae640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fbc1c0714a0 0x7fbc1c073080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:01.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.488+0000 7fbc226ae640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc1c0b07e0 con 0x7fbc1c0b0400 2024-04-25T23:54:01.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.488+0000 7fbc226ae640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc1c072bf0 con 0x7fbc1c0735c0 2024-04-25T23:54:01.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.488+0000 7fbc226ae640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7fbc1c070f80 con 0x7fbc1c0714a0 2024-04-25T23:54:01.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.488+0000 7fbc1bfff640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fbc1c0b0400 0x7fbc1c142f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:35368/0 (socket says 172.21.15.57:35368) 2024-04-25T23:54:01.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.488+0000 7fbc1bfff640 1 -- 172.21.15.57:0/4254599918 learned_addr learned my addr 172.21.15.57:0/4254599918 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-04-25T23:54:01.473 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.488+0000 7fbc1b7fe640 1 --2- 172.21.15.57:0/4254599918 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fbc1c0714a0 0x7fbc1c073080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:01.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.488+0000 7fbc1affd640 1 -- 172.21.15.57:0/4254599918 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4095465766 0 0) 0x7fbc140014a0 con 0x7fbc1c0b0400 2024-04-25T23:54:01.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.488+0000 7fbc1affd640 1 -- 172.21.15.57:0/4254599918 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbc08003530 con 0x7fbc1c0b0400 2024-04-25T23:54:01.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.488+0000 7fbc1b7fe640 1 -- 172.21.15.57:0/4254599918 >> v1:172.21.15.80:6789/0 conn(0x7fbc1c0735c0 legacy=0x7fbc1c13c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:54:01.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.488+0000 7fbc1b7fe640 1 -- 172.21.15.57:0/4254599918 >> v1:172.21.15.57:6789/0 conn(0x7fbc1c0b0400 legacy=0x7fbc1c142f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:01.474 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.488+0000 7fbc1b7fe640 1 -- 172.21.15.57:0/4254599918 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc1c0718e0 con 0x7fbc1c0714a0 2024-04-25T23:54:01.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.492+0000 7fbc1b7fe640 1 --2- 172.21.15.57:0/4254599918 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fbc1c0714a0 0x7fbc1c073080 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbc040097f0 tx=0x7fbc0401c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4085fecf3855b14 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:54:01.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.492+0000 7fbc1affd640 1 -- 172.21.15.57:0/4254599918 <== mon.2 v2:172.21.15.57:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc040b0070 con 0x7fbc1c0714a0 2024-04-25T23:54:01.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.492+0000 7fbc1affd640 1 -- 172.21.15.57:0/4254599918 <== mon.2 v2:172.21.15.57:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc040ab600 con 0x7fbc1c0714a0 2024-04-25T23:54:01.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.492+0000 7fbc1affd640 1 -- 172.21.15.57:0/4254599918 <== mon.2 v2:172.21.15.57:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc040ab9d0 con 0x7fbc1c0714a0 2024-04-25T23:54:01.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.492+0000 7fbc226ae640 1 -- 172.21.15.57:0/4254599918 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fbc1c0714a0 msgr2=0x7fbc1c073080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:01.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.492+0000 7fbc226ae640 1 --2- 172.21.15.57:0/4254599918 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fbc1c0714a0 0x7fbc1c073080 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbc040097f0 tx=0x7fbc0401c450 comp rx=0 tx=0).stop 2024-04-25T23:54:01.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.492+0000 7fbc226ae640 1 -- 172.21.15.57:0/4254599918 shutdown_connections 2024-04-25T23:54:01.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.492+0000 7fbc226ae640 1 --2- 172.21.15.57:0/4254599918 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fbc1c0714a0 0x7fbc1c073080 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:01.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.492+0000 7fbc226ae640 1 -- 172.21.15.57:0/4254599918 >> 172.21.15.57:0/4254599918 conn(0x7fbc1c0b1290 msgr2=0x7fbc1c0b16a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:54:01.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.492+0000 7fbc226ae640 1 -- 172.21.15.57:0/4254599918 shutdown_connections 2024-04-25T23:54:01.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.492+0000 7fbc226ae640 1 -- 172.21.15.57:0/4254599918 wait complete. 2024-04-25T23:54:01.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.492+0000 7fbc226ae640 1 Processor -- start 2024-04-25T23:54:01.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.492+0000 7fbc226ae640 1 -- start start 2024-04-25T23:54:01.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.496+0000 7fbc226ae640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc1c0714a0 0x7fbc1c139590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:01.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.496+0000 7fbc226ae640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fbc1c0735c0 0x7fbc1c139f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:01.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.496+0000 7fbc226ae640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbc1c0b0400 0x7fbc1c13a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:01.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.496+0000 7fbc226ae640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fbc1c14df90 con 0x7fbc1c0b0400 2024-04-25T23:54:01.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.496+0000 7fbc226ae640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fbc1c14de10 con 0x7fbc1c0714a0 2024-04-25T23:54:01.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.496+0000 7fbc226ae640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7fbc1c14e110 con 0x7fbc1c0735c0 2024-04-25T23:54:01.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.496+0000 7fbc1b7fe640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fbc1c0735c0 0x7fbc1c139f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:01.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.496+0000 7fbc1b7fe640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fbc1c0735c0 0x7fbc1c139f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3301/0 says I am v2:172.21.15.57:48674/0 (socket says 172.21.15.57:48674) 2024-04-25T23:54:01.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.496+0000 7fbc1b7fe640 1 -- 172.21.15.57:0/3306466422 learned_addr learned my addr 172.21.15.57:0/3306466422 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:54:01.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.496+0000 7fbc20c4b640 1 --2- 172.21.15.57:0/3306466422 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbc1c0b0400 0x7fbc1c13a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:01.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.496+0000 7fbc1bfff640 1 --2- 172.21.15.57:0/3306466422 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc1c0714a0 0x7fbc1c139590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:01.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.496+0000 7fbc1b7fe640 1 -- 172.21.15.57:0/3306466422 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc1c0714a0 msgr2=0x7fbc1c139590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:01.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.496+0000 7fbc1b7fe640 1 --2- 172.21.15.57:0/3306466422 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc1c0714a0 0x7fbc1c139590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:01.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.496+0000 7fbc1b7fe640 1 -- 172.21.15.57:0/3306466422 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbc1c0b0400 msgr2=0x7fbc1c13a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:01.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.496+0000 7fbc1b7fe640 1 --2- 172.21.15.57:0/3306466422 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbc1c0b0400 0x7fbc1c13a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:01.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.496+0000 7fbc1b7fe640 1 -- 172.21.15.57:0/3306466422 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc1c13b260 con 0x7fbc1c0735c0 2024-04-25T23:54:01.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.496+0000 7fbc1b7fe640 1 --2- 172.21.15.57:0/3306466422 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fbc1c0735c0 0x7fbc1c139f10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbc040097f0 tx=0x7fbc040191f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:54:01.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.496+0000 7fbc18ff9640 1 -- 172.21.15.57:0/3306466422 <== mon.2 v2:172.21.15.57:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc040b8020 con 0x7fbc1c0735c0 2024-04-25T23:54:01.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.496+0000 7fbc18ff9640 1 -- 172.21.15.57:0/3306466422 <== mon.2 v2:172.21.15.57:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc040aa7d0 con 0x7fbc1c0735c0 2024-04-25T23:54:01.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.496+0000 7fbc226ae640 1 -- 172.21.15.57:0/3306466422 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc1c13b540 con 0x7fbc1c0735c0 2024-04-25T23:54:01.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.496+0000 7fbc18ff9640 1 -- 172.21.15.57:0/3306466422 <== mon.2 v2:172.21.15.57:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc040b0040 con 0x7fbc1c0735c0 2024-04-25T23:54:01.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.496+0000 7fbc226ae640 1 -- 172.21.15.57:0/3306466422 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc1c13bf90 con 0x7fbc1c0735c0 2024-04-25T23:54:01.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.496+0000 7fbc226ae640 1 -- 172.21.15.57:0/3306466422 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc1c0761e0 con 0x7fbc1c0735c0 2024-04-25T23:54:01.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.496+0000 7fbc18ff9640 1 -- 172.21.15.57:0/3306466422 <== mon.2 v2:172.21.15.57:3301/0 4 ==== mgrmap(e 5) ==== 87410+0+0 (secure 0 0 0) 0x7fbc040bb020 con 0x7fbc1c0735c0 2024-04-25T23:54:01.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.496+0000 7fbc18ff9640 1 --2- 172.21.15.57:0/3306466422 >> v2:172.21.15.80:6816/2493457677 conn(0x7fbbfc0678b0 0x7fbbfc069d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:01.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.496+0000 7fbc18ff9640 1 -- 172.21.15.57:0/3306466422 <== mon.2 v2:172.21.15.57:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 4525+0+0 (secure 0 0 0) 0x7fbc0411f580 con 0x7fbc1c0735c0 2024-04-25T23:54:01.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.496+0000 7fbc1bfff640 1 --2- 172.21.15.57:0/3306466422 >> v2:172.21.15.80:6816/2493457677 conn(0x7fbbfc0678b0 0x7fbbfc069d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:01.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.500+0000 7fbc1bfff640 1 --2- 172.21.15.57:0/3306466422 >> v2:172.21.15.80:6816/2493457677 conn(0x7fbbfc0678b0 0x7fbbfc069d70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbc14005c10 tx=0x7fbc14009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:54:01.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.504+0000 7fbc18ff9640 1 -- 172.21.15.57:0/3306466422 <== mon.2 v2:172.21.15.57:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fbc040c2030 con 0x7fbc1c0735c0 2024-04-25T23:54:01.666 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.680+0000 7fbc226ae640 1 -- 172.21.15.57:0/3306466422 --> v2:172.21.15.80:6816/2493457677 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7fbc1c068b30 con 0x7fbbfc0678b0 2024-04-25T23:54:01.667 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.680+0000 7fbc18ff9640 1 -- 172.21.15.57:0/3306466422 <== mgr.4100 v2:172.21.15.80:6816/2493457677 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fbc1c068b30 con 0x7fbbfc0678b0 2024-04-25T23:54:01.667 INFO:teuthology.orchestra.run.smithi057.stderr:instructed osd(s) 0 to deep-scrub 2024-04-25T23:54:01.669 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.684+0000 7fbc226ae640 1 -- 172.21.15.57:0/3306466422 >> v2:172.21.15.80:6816/2493457677 conn(0x7fbbfc0678b0 msgr2=0x7fbbfc069d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:01.669 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.684+0000 7fbc226ae640 1 --2- 172.21.15.57:0/3306466422 >> v2:172.21.15.80:6816/2493457677 conn(0x7fbbfc0678b0 0x7fbbfc069d70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbc14005c10 tx=0x7fbc14009040 comp rx=0 tx=0).stop 2024-04-25T23:54:01.669 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.684+0000 7fbc226ae640 1 -- 172.21.15.57:0/3306466422 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fbc1c0735c0 msgr2=0x7fbc1c139f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:01.669 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.684+0000 7fbc226ae640 1 --2- 172.21.15.57:0/3306466422 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fbc1c0735c0 0x7fbc1c139f10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbc040097f0 tx=0x7fbc040191f0 comp rx=0 tx=0).stop 2024-04-25T23:54:01.670 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.684+0000 7fbc226ae640 1 -- 172.21.15.57:0/3306466422 shutdown_connections 2024-04-25T23:54:01.670 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.684+0000 7fbc226ae640 1 --2- 172.21.15.57:0/3306466422 >> v2:172.21.15.80:6816/2493457677 conn(0x7fbbfc0678b0 0x7fbbfc069d70 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:01.670 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.684+0000 7fbc226ae640 1 --2- 172.21.15.57:0/3306466422 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbc1c0b0400 0x7fbc1c13a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:01.670 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.684+0000 7fbc226ae640 1 --2- 172.21.15.57:0/3306466422 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fbc1c0735c0 0x7fbc1c139f10 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:01.670 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.684+0000 7fbc226ae640 1 --2- 172.21.15.57:0/3306466422 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc1c0714a0 0x7fbc1c139590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:01.670 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.684+0000 7fbc226ae640 1 -- 172.21.15.57:0/3306466422 >> 172.21.15.57:0/3306466422 conn(0x7fbc1c0b1290 msgr2=0x7fbc1c0b2690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:54:01.670 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.684+0000 7fbc226ae640 1 -- 172.21.15.57:0/3306466422 shutdown_connections 2024-04-25T23:54:01.670 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.684+0000 7fbc226ae640 1 -- 172.21.15.57:0/3306466422 wait complete. 2024-04-25T23:54:01.686 INFO:tasks.ceph:Scrubbing osd.1 2024-04-25T23:54:01.687 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-25T23:54:01.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.832+0000 7f7be1b46640 1 Processor -- start 2024-04-25T23:54:01.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.832+0000 7f7be1b46640 1 -- start start 2024-04-25T23:54:01.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.832+0000 7f7be1b46640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f7bdc071d60 0x7f7bdc070de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:01.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.832+0000 7f7be1b46640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f7bdc13c690 0x7f7bdc13cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:01.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.832+0000 7f7be1b46640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7bdc142bd0 con 0x7f7bdc06ea70 2024-04-25T23:54:01.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.832+0000 7f7be1b46640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f7bdc072140 con 0x7f7bdc13c690 2024-04-25T23:54:01.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.832+0000 7f7be1b46640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f7bdc142eb0 con 0x7f7bdc071d60 2024-04-25T23:54:01.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.832+0000 7f7bdb7fe640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f7bdc071d60 0x7f7bdc070de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:01.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.832+0000 7f7bdb7fe640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f7bdc071d60 0x7f7bdc070de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3301/0 says I am v2:172.21.15.57:48676/0 (socket says 172.21.15.57:48676) 2024-04-25T23:54:01.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.832+0000 7f7bdb7fe640 1 -- 172.21.15.57:0/1281518678 learned_addr learned my addr 172.21.15.57:0/1281518678 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:54:01.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.832+0000 7f7bdbfff640 1 --2- 172.21.15.57:0/1281518678 >> v2:172.21.15.80:3300/0 conn(0x7f7bdc13c690 0x7f7bdc13cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:01.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.832+0000 7f7bdb7fe640 1 -- 172.21.15.57:0/1281518678 >> v2:172.21.15.80:3300/0 conn(0x7f7bdc13c690 msgr2=0x7f7bdc13cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:54:01.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.832+0000 7f7bdb7fe640 1 --2- 172.21.15.57:0/1281518678 >> v2:172.21.15.80:3300/0 conn(0x7f7bdc13c690 0x7f7bdc13cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:01.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.832+0000 7f7bdb7fe640 1 -- 172.21.15.57:0/1281518678 >> v1:172.21.15.80:6789/0 conn(0x7f7bdc06ea70 legacy=0x7f7bdc1433d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:54:01.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.832+0000 7f7bdb7fe640 1 -- 172.21.15.57:0/1281518678 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bdc071320 con 0x7f7bdc071d60 2024-04-25T23:54:01.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.836+0000 7f7bdb7fe640 1 --2- 172.21.15.57:0/1281518678 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f7bdc071d60 0x7f7bdc070de0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc009870 tx=0x7f7bcc01c500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4319ed8d2a0f1d2f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:54:01.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.836+0000 7f7bda7fc640 1 -- 172.21.15.57:0/1281518678 <== mon.2 v2:172.21.15.57:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bcc0b0070 con 0x7f7bdc071d60 2024-04-25T23:54:01.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.836+0000 7f7bda7fc640 1 -- 172.21.15.57:0/1281518678 <== mon.2 v2:172.21.15.57:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bcc0ab600 con 0x7f7bdc071d60 2024-04-25T23:54:01.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.836+0000 7f7bda7fc640 1 -- 172.21.15.57:0/1281518678 <== mon.2 v2:172.21.15.57:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bcc0ab9d0 con 0x7f7bdc071d60 2024-04-25T23:54:01.820 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.836+0000 7f7be1b46640 1 -- 172.21.15.57:0/1281518678 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f7bdc071d60 msgr2=0x7f7bdc070de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:01.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.836+0000 7f7be1b46640 1 --2- 172.21.15.57:0/1281518678 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f7bdc071d60 0x7f7bdc070de0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc009870 tx=0x7f7bcc01c500 comp rx=0 tx=0).stop 2024-04-25T23:54:01.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.836+0000 7f7be1b46640 1 -- 172.21.15.57:0/1281518678 shutdown_connections 2024-04-25T23:54:01.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.836+0000 7f7be1b46640 1 --2- 172.21.15.57:0/1281518678 >> v2:172.21.15.80:3300/0 conn(0x7f7bdc13c690 0x7f7bdc13cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:01.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.836+0000 7f7be1b46640 1 --2- 172.21.15.57:0/1281518678 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f7bdc071d60 0x7f7bdc070de0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:01.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.836+0000 7f7be1b46640 1 -- 172.21.15.57:0/1281518678 >> 172.21.15.57:0/1281518678 conn(0x7f7bdc0b8d70 msgr2=0x7f7bdc0b9180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:54:01.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.836+0000 7f7be1b46640 1 -- 172.21.15.57:0/1281518678 shutdown_connections 2024-04-25T23:54:01.821 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.836+0000 7f7be1b46640 1 -- 172.21.15.57:0/1281518678 wait complete. 2024-04-25T23:54:01.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.836+0000 7f7be1b46640 1 Processor -- start 2024-04-25T23:54:01.822 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.836+0000 7f7be1b46640 1 -- start start 2024-04-25T23:54:01.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.836+0000 7f7be1b46640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7bdc06ea70 0x7f7bdc1312e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:01.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.836+0000 7f7be1b46640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f7bdc071d60 0x7f7bdc138250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:01.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.836+0000 7f7be1b46640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7bdc13c690 0x7f7bdc138790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:01.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.836+0000 7f7be1b46640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f7bdc14ff10 con 0x7f7bdc13c690 2024-04-25T23:54:01.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.836+0000 7f7be1b46640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f7bdc06f0c0 con 0x7f7bdc06ea70 2024-04-25T23:54:01.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.836+0000 7f7be1b46640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f7bdc150090 con 0x7f7bdc071d60 2024-04-25T23:54:01.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.836+0000 7f7bdaffd640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f7bdc071d60 0x7f7bdc138250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:01.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.836+0000 7f7bdbfff640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7bdc13c690 0x7f7bdc138790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:01.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.836+0000 7f7bdaffd640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f7bdc071d60 0x7f7bdc138250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3301/0 says I am v2:172.21.15.57:48688/0 (socket says 172.21.15.57:48688) 2024-04-25T23:54:01.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.836+0000 7f7bdaffd640 1 -- 172.21.15.57:0/2743377078 learned_addr learned my addr 172.21.15.57:0/2743377078 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:54:01.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.840+0000 7f7bdb7fe640 1 --2- 172.21.15.57:0/2743377078 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7bdc06ea70 0x7f7bdc1312e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:01.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.840+0000 7f7bdbfff640 1 -- 172.21.15.57:0/2743377078 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f7bdc071d60 msgr2=0x7f7bdc138250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:01.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.840+0000 7f7bdbfff640 1 --2- 172.21.15.57:0/2743377078 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f7bdc071d60 0x7f7bdc138250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:01.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.840+0000 7f7bdbfff640 1 -- 172.21.15.57:0/2743377078 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7bdc06ea70 msgr2=0x7f7bdc1312e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:01.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.840+0000 7f7bdbfff640 1 --2- 172.21.15.57:0/2743377078 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7bdc06ea70 0x7f7bdc1312e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:01.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.840+0000 7f7bdbfff640 1 -- 172.21.15.57:0/2743377078 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bdc131d90 con 0x7f7bdc13c690 2024-04-25T23:54:01.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.840+0000 7f7bdaffd640 1 --2- 172.21.15.57:0/2743377078 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f7bdc071d60 0x7f7bdc138250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T23:54:01.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.840+0000 7f7bdbfff640 1 --2- 172.21.15.57:0/2743377078 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7bdc13c690 0x7f7bdc138790 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7bd400cc80 tx=0x7f7bd400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:54:01.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.840+0000 7f7bc3fff640 1 -- 172.21.15.57:0/2743377078 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd400ce30 con 0x7f7bdc13c690 2024-04-25T23:54:01.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.840+0000 7f7bc3fff640 1 -- 172.21.15.57:0/2743377078 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bd40039a0 con 0x7f7bdc13c690 2024-04-25T23:54:01.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.840+0000 7f7be1b46640 1 -- 172.21.15.57:0/2743377078 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bdc130d40 con 0x7f7bdc13c690 2024-04-25T23:54:01.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.840+0000 7f7bc3fff640 1 -- 172.21.15.57:0/2743377078 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd4003d70 con 0x7f7bdc13c690 2024-04-25T23:54:01.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.840+0000 7f7be1b46640 1 -- 172.21.15.57:0/2743377078 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bdc1427f0 con 0x7f7bdc13c690 2024-04-25T23:54:01.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.840+0000 7f7be1b46640 1 -- 172.21.15.57:0/2743377078 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7bdc138cd0 con 0x7f7bdc13c690 2024-04-25T23:54:01.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.840+0000 7f7bc3fff640 1 -- 172.21.15.57:0/2743377078 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) ==== 87410+0+0 (secure 0 0 0) 0x7f7bd4003f10 con 0x7f7bdc13c690 2024-04-25T23:54:01.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.840+0000 7f7bc3fff640 1 --2- 172.21.15.57:0/2743377078 >> v2:172.21.15.80:6816/2493457677 conn(0x7f7bc8067880 0x7f7bc8069d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:01.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.840+0000 7f7bc3fff640 1 -- 172.21.15.57:0/2743377078 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 4525+0+0 (secure 0 0 0) 0x7f7bd4086050 con 0x7f7bdc13c690 2024-04-25T23:54:01.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.840+0000 7f7bc3fff640 1 --2- 172.21.15.57:0/2743377078 >> v2:172.21.15.57:6808/2225240106 conn(0x7f7bc806e5e0 0x7f7bc8070a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:01.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.840+0000 7f7bc3fff640 1 -- 172.21.15.57:0/2743377078 --> v2:172.21.15.57:6808/2225240106 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7bd401ae50 con 0x7f7bc806e5e0 2024-04-25T23:54:01.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.840+0000 7f7bc3fff640 1 -- 172.21.15.57:0/2743377078 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f7bd40864e0 con 0x7f7bdc13c690 2024-04-25T23:54:01.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.840+0000 7f7bdb7fe640 1 --2- 172.21.15.57:0/2743377078 >> v2:172.21.15.80:6816/2493457677 conn(0x7f7bc8067880 0x7f7bc8069d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:01.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.840+0000 7f7bdaffd640 1 --2- 172.21.15.57:0/2743377078 >> v2:172.21.15.57:6808/2225240106 conn(0x7f7bc806e5e0 0x7f7bc8070a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:01.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.840+0000 7f7bdaffd640 1 --2- 172.21.15.57:0/2743377078 >> v2:172.21.15.57:6808/2225240106 conn(0x7f7bc806e5e0 0x7f7bc8070a00 crc :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:54:01.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.840+0000 7f7bdb7fe640 1 --2- 172.21.15.57:0/2743377078 >> v2:172.21.15.80:6816/2493457677 conn(0x7f7bc8067880 0x7f7bc8069d40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc0ad000 tx=0x7f7bcc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:54:01.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.844+0000 7f7bc3fff640 1 -- 172.21.15.57:0/2743377078 <== osd.1 v2:172.21.15.57:6808/2225240106 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7bd401ae50 con 0x7f7bc806e5e0 2024-04-25T23:54:01.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.856+0000 7f7be1b46640 1 -- 172.21.15.57:0/2743377078 --> v2:172.21.15.57:6808/2225240106 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f7bdc0761e0 con 0x7f7bc806e5e0 2024-04-25T23:54:01.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.856+0000 7f7bc3fff640 1 -- 172.21.15.57:0/2743377078 <== osd.1 v2:172.21.15.57:6808/2225240106 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f7bdc0761e0 con 0x7f7bc806e5e0 2024-04-25T23:54:01.841 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-25T23:54:01.841 INFO:teuthology.orchestra.run.smithi057.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T23:54:01.841 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-25T23:54:01.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.856+0000 7f7be1b46640 1 -- 172.21.15.57:0/2743377078 >> v2:172.21.15.57:6808/2225240106 conn(0x7f7bc806e5e0 msgr2=0x7f7bc8070a00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:01.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.856+0000 7f7be1b46640 1 --2- 172.21.15.57:0/2743377078 >> v2:172.21.15.57:6808/2225240106 conn(0x7f7bc806e5e0 0x7f7bc8070a00 crc :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:01.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.856+0000 7f7be1b46640 1 -- 172.21.15.57:0/2743377078 >> v2:172.21.15.80:6816/2493457677 conn(0x7f7bc8067880 msgr2=0x7f7bc8069d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:01.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.856+0000 7f7be1b46640 1 --2- 172.21.15.57:0/2743377078 >> v2:172.21.15.80:6816/2493457677 conn(0x7f7bc8067880 0x7f7bc8069d40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc0ad000 tx=0x7f7bcc008040 comp rx=0 tx=0).stop 2024-04-25T23:54:01.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.856+0000 7f7be1b46640 1 -- 172.21.15.57:0/2743377078 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7bdc13c690 msgr2=0x7f7bdc138790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:01.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.856+0000 7f7be1b46640 1 --2- 172.21.15.57:0/2743377078 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7bdc13c690 0x7f7bdc138790 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7bd400cc80 tx=0x7f7bd400a5c0 comp rx=0 tx=0).stop 2024-04-25T23:54:01.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.856+0000 7f7bdbfff640 1 -- 172.21.15.57:0/2743377078 reap_dead start 2024-04-25T23:54:01.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.856+0000 7f7be1b46640 1 -- 172.21.15.57:0/2743377078 shutdown_connections 2024-04-25T23:54:01.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.856+0000 7f7be1b46640 1 -- 172.21.15.57:0/2743377078 >> 172.21.15.57:0/2743377078 conn(0x7f7bdc0b8d70 msgr2=0x7f7bdc0b9150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:54:01.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.856+0000 7f7be1b46640 1 -- 172.21.15.57:0/2743377078 shutdown_connections 2024-04-25T23:54:01.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:01.856+0000 7f7be1b46640 1 -- 172.21.15.57:0/2743377078 wait complete. 2024-04-25T23:54:01.857 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-25T23:54:01.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.000+0000 7f896b5c0640 1 Processor -- start 2024-04-25T23:54:01.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.000+0000 7f896b5c0640 1 -- start start 2024-04-25T23:54:01.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.000+0000 7f896b5c0640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f89640b2fa0 0x7f89641463d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:01.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.000+0000 7f896b5c0640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f896406f0d0 con 0x7f896406ecf0 2024-04-25T23:54:01.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.000+0000 7f896b5c0640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f896406f7e0 con 0x7f8964146910 2024-04-25T23:54:01.986 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.000+0000 7f896b5c0640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f8964070020 con 0x7f89640b2fa0 2024-04-25T23:54:01.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.000+0000 7f8968b5b640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f89640b2fa0 0x7f89641463d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:01.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.000+0000 7f896935c640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f896406ecf0 0x7f8964142290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:35374/0 (socket says 172.21.15.57:35374) 2024-04-25T23:54:01.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.000+0000 7f896935c640 1 -- 172.21.15.57:0/2783906809 learned_addr learned my addr 172.21.15.57:0/2783906809 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-04-25T23:54:01.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.000+0000 7f8968b5b640 1 -- 172.21.15.57:0/2783906809 >> v1:172.21.15.80:6789/0 conn(0x7f8964146910 legacy=0x7f896414d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:54:01.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.000+0000 7f8968b5b640 1 -- 172.21.15.57:0/2783906809 >> v1:172.21.15.57:6789/0 conn(0x7f896406ecf0 legacy=0x7f8964142290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:01.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.000+0000 7f8968b5b640 1 -- 172.21.15.57:0/2783906809 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f896406fbe0 con 0x7f89640b2fa0 2024-04-25T23:54:01.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.000+0000 7f895bfff640 1 -- 172.21.15.57:0/2783906809 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 938554436 0 0) 0x7f8960001120 con 0x7f896406ecf0 2024-04-25T23:54:01.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.004+0000 7f8968b5b640 1 --2- 172.21.15.57:0/2783906809 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f89640b2fa0 0x7f89641463d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f89540097f0 tx=0x7f895401c3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84889748b4e827f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:54:01.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.004+0000 7f895bfff640 1 -- 172.21.15.57:0/2783906809 <== mon.2 v2:172.21.15.57:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89540b0070 con 0x7f89640b2fa0 2024-04-25T23:54:01.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.004+0000 7f895bfff640 1 -- 172.21.15.57:0/2783906809 <== mon.2 v2:172.21.15.57:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89540ab600 con 0x7f89640b2fa0 2024-04-25T23:54:01.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.004+0000 7f895bfff640 1 -- 172.21.15.57:0/2783906809 <== mon.2 v2:172.21.15.57:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89540ab9d0 con 0x7f89640b2fa0 2024-04-25T23:54:01.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.004+0000 7f896b5c0640 1 -- 172.21.15.57:0/2783906809 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f89640b2fa0 msgr2=0x7f89641463d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:01.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.004+0000 7f896b5c0640 1 --2- 172.21.15.57:0/2783906809 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f89640b2fa0 0x7f89641463d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f89540097f0 tx=0x7f895401c3f0 comp rx=0 tx=0).stop 2024-04-25T23:54:01.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.004+0000 7f896b5c0640 1 -- 172.21.15.57:0/2783906809 shutdown_connections 2024-04-25T23:54:01.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.004+0000 7f896b5c0640 1 --2- 172.21.15.57:0/2783906809 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f89640b2fa0 0x7f89641463d0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:01.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.004+0000 7f896b5c0640 1 -- 172.21.15.57:0/2783906809 >> 172.21.15.57:0/2783906809 conn(0x7f89640b8f00 msgr2=0x7f89640b9310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:54:01.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.004+0000 7f896b5c0640 1 -- 172.21.15.57:0/2783906809 shutdown_connections 2024-04-25T23:54:01.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.004+0000 7f896b5c0640 1 -- 172.21.15.57:0/2783906809 wait complete. 2024-04-25T23:54:01.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.004+0000 7f896b5c0640 1 Processor -- start 2024-04-25T23:54:01.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.004+0000 7f896b5c0640 1 -- start start 2024-04-25T23:54:01.994 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.004+0000 7f896b5c0640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f896406ecf0 0x7f896413ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:01.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.004+0000 7f896b5c0640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f89640b2fa0 0x7f896413e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:01.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.004+0000 7f896b5c0640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8964146910 0x7f896413e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:01.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.004+0000 7f896b5c0640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f8964151750 con 0x7f8964146910 2024-04-25T23:54:01.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.004+0000 7f896b5c0640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8964147180 con 0x7f896406ecf0 2024-04-25T23:54:01.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.004+0000 7f896b5c0640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f89641518d0 con 0x7f89640b2fa0 2024-04-25T23:54:01.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.004+0000 7f8969b5d640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8964146910 0x7f896413e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:01.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.004+0000 7f8968b5b640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f89640b2fa0 0x7f896413e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:01.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.004+0000 7f8969b5d640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8964146910 0x7f896413e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37200/0 (socket says 172.21.15.57:37200) 2024-04-25T23:54:01.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.004+0000 7f8968b5b640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f89640b2fa0 0x7f896413e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3301/0 says I am v2:172.21.15.57:48714/0 (socket says 172.21.15.57:48714) 2024-04-25T23:54:01.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.004+0000 7f8969b5d640 1 -- 172.21.15.57:0/613550407 learned_addr learned my addr 172.21.15.57:0/613550407 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:54:01.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.004+0000 7f896935c640 1 --2- 172.21.15.57:0/613550407 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f896406ecf0 0x7f896413ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:01.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.004+0000 7f8969b5d640 1 -- 172.21.15.57:0/613550407 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f89640b2fa0 msgr2=0x7f896413e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:01.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.004+0000 7f8969b5d640 1 --2- 172.21.15.57:0/613550407 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f89640b2fa0 0x7f896413e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:01.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.004+0000 7f8969b5d640 1 -- 172.21.15.57:0/613550407 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f896406ecf0 msgr2=0x7f896413ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:01.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.004+0000 7f8969b5d640 1 --2- 172.21.15.57:0/613550407 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f896406ecf0 0x7f896413ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:01.995 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.004+0000 7f8969b5d640 1 -- 172.21.15.57:0/613550407 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f896413ef10 con 0x7f8964146910 2024-04-25T23:54:01.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.008+0000 7f8969b5d640 1 --2- 172.21.15.57:0/613550407 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8964146910 0x7f896413e9d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f895c00b970 tx=0x7f895c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:54:01.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.008+0000 7f8959ffb640 1 -- 172.21.15.57:0/613550407 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f895c019070 con 0x7f8964146910 2024-04-25T23:54:01.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.008+0000 7f8959ffb640 1 -- 172.21.15.57:0/613550407 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f895c00cb40 con 0x7f8964146910 2024-04-25T23:54:01.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.008+0000 7f8959ffb640 1 -- 172.21.15.57:0/613550407 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f895c014440 con 0x7f8964146910 2024-04-25T23:54:01.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.008+0000 7f896b5c0640 1 -- 172.21.15.57:0/613550407 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8964139900 con 0x7f8964146910 2024-04-25T23:54:01.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.008+0000 7f896b5c0640 1 -- 172.21.15.57:0/613550407 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89640ad850 con 0x7f8964146910 2024-04-25T23:54:01.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.008+0000 7f896b5c0640 1 -- 172.21.15.57:0/613550407 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89640761e0 con 0x7f8964146910 2024-04-25T23:54:01.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.008+0000 7f8959ffb640 1 -- 172.21.15.57:0/613550407 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) ==== 87410+0+0 (secure 0 0 0) 0x7f895c01c070 con 0x7f8964146910 2024-04-25T23:54:01.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.008+0000 7f8959ffb640 1 --2- 172.21.15.57:0/613550407 >> v2:172.21.15.80:6816/2493457677 conn(0x7f89480678b0 0x7f8948069d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:01.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.008+0000 7f8959ffb640 1 -- 172.21.15.57:0/613550407 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 4525+0+0 (secure 0 0 0) 0x7f895c085310 con 0x7f8964146910 2024-04-25T23:54:01.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.008+0000 7f896935c640 1 --2- 172.21.15.57:0/613550407 >> v2:172.21.15.80:6816/2493457677 conn(0x7f89480678b0 0x7f8948069d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:01.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.008+0000 7f896935c640 1 --2- 172.21.15.57:0/613550407 >> v2:172.21.15.80:6816/2493457677 conn(0x7f89480678b0 0x7f8948069d70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8960007fd0 tx=0x7f8960007980 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:54:01.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.012+0000 7f8959ffb640 1 -- 172.21.15.57:0/613550407 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f895c055c90 con 0x7f8964146910 2024-04-25T23:54:02.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.168+0000 7f896b5c0640 1 -- 172.21.15.57:0/613550407 --> v2:172.21.15.80:6816/2493457677 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f8964068b70 con 0x7f89480678b0 2024-04-25T23:54:02.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.168+0000 7f8959ffb640 1 -- 172.21.15.57:0/613550407 <== mgr.4100 v2:172.21.15.80:6816/2493457677 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f8964068b70 con 0x7f89480678b0 2024-04-25T23:54:02.153 INFO:teuthology.orchestra.run.smithi057.stderr:instructed osd(s) 1 to deep-scrub 2024-04-25T23:54:02.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.172+0000 7f896b5c0640 1 -- 172.21.15.57:0/613550407 >> v2:172.21.15.80:6816/2493457677 conn(0x7f89480678b0 msgr2=0x7f8948069d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:02.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.172+0000 7f896b5c0640 1 --2- 172.21.15.57:0/613550407 >> v2:172.21.15.80:6816/2493457677 conn(0x7f89480678b0 0x7f8948069d70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8960007fd0 tx=0x7f8960007980 comp rx=0 tx=0).stop 2024-04-25T23:54:02.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.172+0000 7f896b5c0640 1 -- 172.21.15.57:0/613550407 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8964146910 msgr2=0x7f896413e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:02.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.172+0000 7f896b5c0640 1 --2- 172.21.15.57:0/613550407 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8964146910 0x7f896413e9d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f895c00b970 tx=0x7f895c00be30 comp rx=0 tx=0).stop 2024-04-25T23:54:02.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.172+0000 7f896b5c0640 1 -- 172.21.15.57:0/613550407 shutdown_connections 2024-04-25T23:54:02.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.172+0000 7f896b5c0640 1 --2- 172.21.15.57:0/613550407 >> v2:172.21.15.80:6816/2493457677 conn(0x7f89480678b0 0x7f8948069d70 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:02.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.172+0000 7f896b5c0640 1 --2- 172.21.15.57:0/613550407 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8964146910 0x7f896413e9d0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:02.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.172+0000 7f896b5c0640 1 --2- 172.21.15.57:0/613550407 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f89640b2fa0 0x7f896413e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:02.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.172+0000 7f896b5c0640 1 --2- 172.21.15.57:0/613550407 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f896406ecf0 0x7f896413ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:02.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.172+0000 7f896b5c0640 1 -- 172.21.15.57:0/613550407 >> 172.21.15.57:0/613550407 conn(0x7f89640b8f00 msgr2=0x7f89640b92e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:54:02.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.172+0000 7f896b5c0640 1 -- 172.21.15.57:0/613550407 shutdown_connections 2024-04-25T23:54:02.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.172+0000 7f896b5c0640 1 -- 172.21.15.57:0/613550407 wait complete. 2024-04-25T23:54:02.171 INFO:tasks.ceph:Scrubbing osd.2 2024-04-25T23:54:02.171 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-25T23:54:02.292 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.308+0000 7f31bbe61640 1 Processor -- start 2024-04-25T23:54:02.292 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.308+0000 7f31bbe61640 1 -- start start 2024-04-25T23:54:02.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.308+0000 7f31bbe61640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f31b406fd40 0x7f31b4140390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:02.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.308+0000 7f31bbe61640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f31b413f8c0 0x7f31b4135fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:02.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.308+0000 7f31bbe61640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31b406d280 con 0x7f31b41364e0 2024-04-25T23:54:02.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.308+0000 7f31bbe61640 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 -- 0x7f31b4070120 con 0x7f31b406fd40 2024-04-25T23:54:02.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.308+0000 7f31bbe61640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f31b40b1120 con 0x7f31b413f8c0 2024-04-25T23:54:02.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.308+0000 7f31b9bfd640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f31b406fd40 0x7f31b4140390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:02.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.308+0000 7f31b93fc640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f31b413f8c0 0x7f31b4135fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:02.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.308+0000 7f31b9bfd640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f31b406fd40 0x7f31b4140390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37216/0 (socket says 172.21.15.57:37216) 2024-04-25T23:54:02.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.308+0000 7f31b9bfd640 1 -- 172.21.15.57:0/1965240956 learned_addr learned my addr 172.21.15.57:0/1965240956 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:54:02.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.308+0000 7f31b9bfd640 1 -- 172.21.15.57:0/1965240956 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f31b413f8c0 msgr2=0x7f31b4135fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:54:02.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.308+0000 7f31b9bfd640 1 --2- 172.21.15.57:0/1965240956 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f31b413f8c0 0x7f31b4135fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:02.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.308+0000 7f31b9bfd640 1 -- 172.21.15.57:0/1965240956 >> v1:172.21.15.57:6789/0 conn(0x7f31b41364e0 legacy=0x7f31b4071420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:02.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.308+0000 7f31b9bfd640 1 -- 172.21.15.57:0/1965240956 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31b413fe70 con 0x7f31b406fd40 2024-04-25T23:54:02.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.308+0000 7f31b8bfb640 1 -- 172.21.15.57:0/1965240956 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 18804822 0 0) 0x7f31b406d280 con 0x7f31b41364e0 2024-04-25T23:54:02.295 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.308+0000 7f31b93fc640 1 --2- 172.21.15.57:0/1965240956 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f31b413f8c0 0x7f31b4135fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T23:54:02.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.312+0000 7f31b9bfd640 1 --2- 172.21.15.57:0/1965240956 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f31b406fd40 0x7f31b4140390 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f31ac009e50 tx=0x7f31ac01ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cf13db9d054724b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:54:02.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.312+0000 7f31b8bfb640 1 -- 172.21.15.57:0/1965240956 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ac0a57f0 con 0x7f31b406fd40 2024-04-25T23:54:02.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.312+0000 7f31b8bfb640 1 -- 172.21.15.57:0/1965240956 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31ac0a5990 con 0x7f31b406fd40 2024-04-25T23:54:02.296 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.312+0000 7f31b8bfb640 1 -- 172.21.15.57:0/1965240956 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ac0a5d60 con 0x7f31b406fd40 2024-04-25T23:54:02.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.312+0000 7f31bbe61640 1 -- 172.21.15.57:0/1965240956 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f31b406fd40 msgr2=0x7f31b4140390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:02.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.312+0000 7f31bbe61640 1 --2- 172.21.15.57:0/1965240956 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f31b406fd40 0x7f31b4140390 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f31ac009e50 tx=0x7f31ac01ca10 comp rx=0 tx=0).stop 2024-04-25T23:54:02.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.312+0000 7f31bbe61640 1 -- 172.21.15.57:0/1965240956 shutdown_connections 2024-04-25T23:54:02.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.312+0000 7f31bbe61640 1 --2- 172.21.15.57:0/1965240956 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f31b413f8c0 0x7f31b4135fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:02.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.312+0000 7f31bbe61640 1 --2- 172.21.15.57:0/1965240956 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f31b406fd40 0x7f31b4140390 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:02.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.312+0000 7f31bbe61640 1 -- 172.21.15.57:0/1965240956 >> 172.21.15.57:0/1965240956 conn(0x7f31b40b1390 msgr2=0x7f31b40b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:54:02.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.312+0000 7f31bbe61640 1 -- 172.21.15.57:0/1965240956 shutdown_connections 2024-04-25T23:54:02.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.312+0000 7f31bbe61640 1 -- 172.21.15.57:0/1965240956 wait complete. 2024-04-25T23:54:02.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.312+0000 7f31bbe61640 1 Processor -- start 2024-04-25T23:54:02.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.312+0000 7f31bbe61640 1 -- start start 2024-04-25T23:54:02.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.312+0000 7f31bbe61640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f31b406fd40 0x7f31b414e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:02.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.312+0000 7f31bbe61640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f31b41364e0 0x7f31b41db0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:02.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.312+0000 7f31bbe61640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f31b413f8c0 0x7f31b412dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:02.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.312+0000 7f31bbe61640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f31b40706e0 con 0x7f31b413f8c0 2024-04-25T23:54:02.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.312+0000 7f31bbe61640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f31b4070560 con 0x7f31b41364e0 2024-04-25T23:54:02.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.312+0000 7f31bbe61640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f31b4070860 con 0x7f31b406fd40 2024-04-25T23:54:02.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.316+0000 7f31b9bfd640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f31b406fd40 0x7f31b414e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:02.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.316+0000 7f31b9bfd640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f31b406fd40 0x7f31b414e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3301/0 says I am v2:172.21.15.57:48734/0 (socket says 172.21.15.57:48734) 2024-04-25T23:54:02.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.316+0000 7f31b9bfd640 1 -- 172.21.15.57:0/1036030189 learned_addr learned my addr 172.21.15.57:0/1036030189 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:54:02.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.316+0000 7f31ba3fe640 1 --2- 172.21.15.57:0/1036030189 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f31b413f8c0 0x7f31b412dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:02.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.316+0000 7f31b93fc640 1 --2- 172.21.15.57:0/1036030189 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f31b41364e0 0x7f31b41db0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:02.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.316+0000 7f31b9bfd640 1 -- 172.21.15.57:0/1036030189 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f31b41364e0 msgr2=0x7f31b41db0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:02.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.316+0000 7f31b9bfd640 1 --2- 172.21.15.57:0/1036030189 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f31b41364e0 0x7f31b41db0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:02.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.316+0000 7f31b9bfd640 1 -- 172.21.15.57:0/1036030189 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f31b413f8c0 msgr2=0x7f31b412dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:02.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.316+0000 7f31b9bfd640 1 --2- 172.21.15.57:0/1036030189 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f31b413f8c0 0x7f31b412dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:02.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.316+0000 7f31b9bfd640 1 -- 172.21.15.57:0/1036030189 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31b41db600 con 0x7f31b406fd40 2024-04-25T23:54:02.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.316+0000 7f31b9bfd640 1 --2- 172.21.15.57:0/1036030189 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f31b406fd40 0x7f31b414e5f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f31ac01a640 tx=0x7f31ac0197b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:54:02.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.316+0000 7f31aa7fc640 1 -- 172.21.15.57:0/1036030189 <== mon.2 v2:172.21.15.57:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ac0bc020 con 0x7f31b406fd40 2024-04-25T23:54:02.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.316+0000 7f31aa7fc640 1 -- 172.21.15.57:0/1036030189 <== mon.2 v2:172.21.15.57:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31ac0b8670 con 0x7f31b406fd40 2024-04-25T23:54:02.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.316+0000 7f31bbe61640 1 -- 172.21.15.57:0/1036030189 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31b414c9f0 con 0x7f31b406fd40 2024-04-25T23:54:02.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.316+0000 7f31aa7fc640 1 -- 172.21.15.57:0/1036030189 <== mon.2 v2:172.21.15.57:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ac0b8a60 con 0x7f31b406fd40 2024-04-25T23:54:02.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.316+0000 7f31bbe61640 1 -- 172.21.15.57:0/1036030189 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f31b412e3a0 con 0x7f31b406fd40 2024-04-25T23:54:02.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.316+0000 7f31bbe61640 1 -- 172.21.15.57:0/1036030189 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f31b414dc90 con 0x7f31b406fd40 2024-04-25T23:54:02.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.316+0000 7f31aa7fc640 1 -- 172.21.15.57:0/1036030189 <== mon.2 v2:172.21.15.57:3301/0 4 ==== mgrmap(e 5) ==== 87410+0+0 (secure 0 0 0) 0x7f31ac0bf020 con 0x7f31b406fd40 2024-04-25T23:54:02.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.316+0000 7f31aa7fc640 1 --2- 172.21.15.57:0/1036030189 >> v2:172.21.15.80:6816/2493457677 conn(0x7f31980678b0 0x7f3198069d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:02.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.316+0000 7f31aa7fc640 1 -- 172.21.15.57:0/1036030189 <== mon.2 v2:172.21.15.57:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 4525+0+0 (secure 0 0 0) 0x7f31ac120610 con 0x7f31b406fd40 2024-04-25T23:54:02.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.316+0000 7f31aa7fc640 1 --2- 172.21.15.57:0/1036030189 >> v2:172.21.15.57:6804/1976583347 conn(0x7f319806e610 0x7f3198070a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:02.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.316+0000 7f31aa7fc640 1 -- 172.21.15.57:0/1036030189 --> v2:172.21.15.57:6804/1976583347 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f31980710a0 con 0x7f319806e610 2024-04-25T23:54:02.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.316+0000 7f31aa7fc640 1 -- 172.21.15.57:0/1036030189 <== mon.2 v2:172.21.15.57:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f31ac127050 con 0x7f31b406fd40 2024-04-25T23:54:02.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.316+0000 7f31ba3fe640 1 --2- 172.21.15.57:0/1036030189 >> v2:172.21.15.57:6804/1976583347 conn(0x7f319806e610 0x7f3198070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:02.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.316+0000 7f31ba3fe640 1 -- 172.21.15.57:0/1036030189 >> v2:172.21.15.57:6804/1976583347 conn(0x7f319806e610 msgr2=0x7f3198070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-04-25T23:54:02.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.316+0000 7f31ba3fe640 1 -- 172.21.15.57:0/1036030189 >> v2:172.21.15.57:6804/1976583347 conn(0x7f319806e610 msgr2=0x7f3198070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T23:54:02.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.316+0000 7f31ba3fe640 1 --2- 172.21.15.57:0/1036030189 >> v2:172.21.15.57:6804/1976583347 conn(0x7f319806e610 0x7f3198070a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T23:54:02.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.316+0000 7f31ba3fe640 1 --2- 172.21.15.57:0/1036030189 >> v2:172.21.15.57:6804/1976583347 conn(0x7f319806e610 0x7f3198070a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T23:54:02.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.316+0000 7f31b93fc640 1 --2- 172.21.15.57:0/1036030189 >> v2:172.21.15.80:6816/2493457677 conn(0x7f31980678b0 0x7f3198069d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:02.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.316+0000 7f31b93fc640 1 -- 172.21.15.57:0/1036030189 >> v2:172.21.15.80:6816/2493457677 conn(0x7f31980678b0 msgr2=0x7f3198069d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-04-25T23:54:02.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.316+0000 7f31b93fc640 1 -- 172.21.15.57:0/1036030189 >> v2:172.21.15.80:6816/2493457677 conn(0x7f31980678b0 msgr2=0x7f3198069d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T23:54:02.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.316+0000 7f31b93fc640 1 --2- 172.21.15.57:0/1036030189 >> v2:172.21.15.80:6816/2493457677 conn(0x7f31980678b0 0x7f3198069d70 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0x7f31b41df1c0 tx=0x7f31a4008040 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T23:54:02.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.316+0000 7f31b93fc640 1 --2- 172.21.15.57:0/1036030189 >> v2:172.21.15.80:6816/2493457677 conn(0x7f31980678b0 0x7f3198069d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T23:54:02.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.520+0000 7f31ba3fe640 1 --2- 172.21.15.57:0/1036030189 >> v2:172.21.15.57:6804/1976583347 conn(0x7f319806e610 0x7f3198070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:02.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.520+0000 7f31b93fc640 1 --2- 172.21.15.57:0/1036030189 >> v2:172.21.15.80:6816/2493457677 conn(0x7f31980678b0 0x7f3198069d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:02.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.520+0000 7f31ba3fe640 1 --2- 172.21.15.57:0/1036030189 >> v2:172.21.15.57:6804/1976583347 conn(0x7f319806e610 0x7f3198070a30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:54:02.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.520+0000 7f31b93fc640 1 --2- 172.21.15.57:0/1036030189 >> v2:172.21.15.80:6816/2493457677 conn(0x7f31980678b0 0x7f3198069d70 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f31b41df1c0 tx=0x7f31a4017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:54:02.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.520+0000 7f31aa7fc640 1 -- 172.21.15.57:0/1036030189 <== osd.2 v2:172.21.15.57:6804/1976583347 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f31980710a0 con 0x7f319806e610 2024-04-25T23:54:02.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.540+0000 7f31bbe61640 1 -- 172.21.15.57:0/1036030189 --> v2:172.21.15.57:6804/1976583347 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f31b40761e0 con 0x7f319806e610 2024-04-25T23:54:02.527 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.540+0000 7f31aa7fc640 1 -- 172.21.15.57:0/1036030189 <== osd.2 v2:172.21.15.57:6804/1976583347 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f31b40761e0 con 0x7f319806e610 2024-04-25T23:54:02.527 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-25T23:54:02.527 INFO:teuthology.orchestra.run.smithi057.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T23:54:02.527 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-25T23:54:02.528 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.544+0000 7f31bbe61640 1 -- 172.21.15.57:0/1036030189 >> v2:172.21.15.57:6804/1976583347 conn(0x7f319806e610 msgr2=0x7f3198070a30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:02.528 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.544+0000 7f31bbe61640 1 --2- 172.21.15.57:0/1036030189 >> v2:172.21.15.57:6804/1976583347 conn(0x7f319806e610 0x7f3198070a30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:02.528 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.544+0000 7f31bbe61640 1 -- 172.21.15.57:0/1036030189 >> v2:172.21.15.80:6816/2493457677 conn(0x7f31980678b0 msgr2=0x7f3198069d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:02.528 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.544+0000 7f31bbe61640 1 --2- 172.21.15.57:0/1036030189 >> v2:172.21.15.80:6816/2493457677 conn(0x7f31980678b0 0x7f3198069d70 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f31b41df1c0 tx=0x7f31a4017000 comp rx=0 tx=0).stop 2024-04-25T23:54:02.528 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.544+0000 7f31bbe61640 1 -- 172.21.15.57:0/1036030189 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f31b406fd40 msgr2=0x7f31b414e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:02.528 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.544+0000 7f31bbe61640 1 --2- 172.21.15.57:0/1036030189 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f31b406fd40 0x7f31b414e5f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f31ac01a640 tx=0x7f31ac0197b0 comp rx=0 tx=0).stop 2024-04-25T23:54:02.528 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.544+0000 7f31ba3fe640 1 -- 172.21.15.57:0/1036030189 reap_dead start 2024-04-25T23:54:02.528 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.544+0000 7f31bbe61640 1 -- 172.21.15.57:0/1036030189 shutdown_connections 2024-04-25T23:54:02.528 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.544+0000 7f31bbe61640 1 -- 172.21.15.57:0/1036030189 >> 172.21.15.57:0/1036030189 conn(0x7f31b40b1390 msgr2=0x7f31b40b1770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:54:02.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.544+0000 7f31bbe61640 1 -- 172.21.15.57:0/1036030189 shutdown_connections 2024-04-25T23:54:02.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.544+0000 7f31bbe61640 1 -- 172.21.15.57:0/1036030189 wait complete. 2024-04-25T23:54:02.550 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-25T23:54:02.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.716+0000 7fb764b0c640 1 Processor -- start 2024-04-25T23:54:02.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.716+0000 7fb764b0c640 1 -- start start 2024-04-25T23:54:02.703 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.720+0000 7fb764b0c640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fb760071d00 0x7fb76006fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:02.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.720+0000 7fb764b0c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb76013f500 0x7fb760140760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:02.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.720+0000 7fb764b0c640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7600720e0 con 0x7fb760140340 2024-04-25T23:54:02.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.720+0000 7fb764b0c640 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 -- 0x7fb76013fe20 con 0x7fb760071d00 2024-04-25T23:54:02.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.720+0000 7fb764b0c640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fb760072b10 con 0x7fb76013f500 2024-04-25T23:54:02.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.720+0000 7fb75e59c640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fb760071d00 0x7fb76006fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:02.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.720+0000 7fb75e59c640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fb760071d00 0x7fb76006fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37232/0 (socket says 172.21.15.57:37232) 2024-04-25T23:54:02.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.720+0000 7fb75e59c640 1 -- 172.21.15.57:0/2949266675 learned_addr learned my addr 172.21.15.57:0/2949266675 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:54:02.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.720+0000 7fb75ed9d640 1 --2- 172.21.15.57:0/2949266675 >> v2:172.21.15.80:3300/0 conn(0x7fb76013f500 0x7fb760140760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:02.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.720+0000 7fb75e59c640 1 -- 172.21.15.57:0/2949266675 >> v2:172.21.15.80:3300/0 conn(0x7fb76013f500 msgr2=0x7fb760140760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:54:02.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.720+0000 7fb75e59c640 1 --2- 172.21.15.57:0/2949266675 >> v2:172.21.15.80:3300/0 conn(0x7fb76013f500 0x7fb760140760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:02.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.720+0000 7fb75e59c640 1 -- 172.21.15.57:0/2949266675 >> v1:172.21.15.80:6789/0 conn(0x7fb760140340 legacy=0x7fb76013edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:54:02.704 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.720+0000 7fb75e59c640 1 -- 172.21.15.57:0/2949266675 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb76013fa40 con 0x7fb760071d00 2024-04-25T23:54:02.706 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.720+0000 7fb75e59c640 1 --2- 172.21.15.57:0/2949266675 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb760071d00 0x7fb76006fbd0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb7600728a0 tx=0x7fb75001e920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1490bfd584020af6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:54:02.706 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.720+0000 7fb75dd9b640 1 -- 172.21.15.57:0/2949266675 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7500b4070 con 0x7fb760071d00 2024-04-25T23:54:02.706 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.720+0000 7fb75dd9b640 1 -- 172.21.15.57:0/2949266675 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7500a7800 con 0x7fb760071d00 2024-04-25T23:54:02.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.720+0000 7fb75dd9b640 1 -- 172.21.15.57:0/2949266675 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7500a7bd0 con 0x7fb760071d00 2024-04-25T23:54:02.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.724+0000 7fb764b0c640 1 -- 172.21.15.57:0/2949266675 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb760071d00 msgr2=0x7fb76006fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:02.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.724+0000 7fb764b0c640 1 --2- 172.21.15.57:0/2949266675 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb760071d00 0x7fb76006fbd0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb7600728a0 tx=0x7fb75001e920 comp rx=0 tx=0).stop 2024-04-25T23:54:02.707 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.724+0000 7fb764b0c640 1 -- 172.21.15.57:0/2949266675 shutdown_connections 2024-04-25T23:54:02.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.724+0000 7fb764b0c640 1 --2- 172.21.15.57:0/2949266675 >> v2:172.21.15.80:3300/0 conn(0x7fb76013f500 0x7fb760140760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:02.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.724+0000 7fb764b0c640 1 --2- 172.21.15.57:0/2949266675 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb760071d00 0x7fb76006fbd0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:02.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.724+0000 7fb764b0c640 1 -- 172.21.15.57:0/2949266675 >> 172.21.15.57:0/2949266675 conn(0x7fb7600b5a70 msgr2=0x7fb7600b5e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:54:02.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.724+0000 7fb764b0c640 1 -- 172.21.15.57:0/2949266675 shutdown_connections 2024-04-25T23:54:02.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.724+0000 7fb764b0c640 1 -- 172.21.15.57:0/2949266675 wait complete. 2024-04-25T23:54:02.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.724+0000 7fb764b0c640 1 Processor -- start 2024-04-25T23:54:02.708 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.724+0000 7fb764b0c640 1 -- start start 2024-04-25T23:54:02.710 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.724+0000 7fb764b0c640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb760071d00 0x7fb7601da130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:02.710 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.724+0000 7fb764b0c640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fb76013f500 0x7fb760135ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:02.710 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.724+0000 7fb764b0c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb760140340 0x7fb760136d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:02.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.724+0000 7fb764b0c640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fb760075490 con 0x7fb760071d00 2024-04-25T23:54:02.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.724+0000 7fb764b0c640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb760075310 con 0x7fb760140340 2024-04-25T23:54:02.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.724+0000 7fb764b0c640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7fb760075610 con 0x7fb76013f500 2024-04-25T23:54:02.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.724+0000 7fb755d9b640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fb76013f500 0x7fb760135ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:02.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.724+0000 7fb755d9b640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fb76013f500 0x7fb760135ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3301/0 says I am v2:172.21.15.57:48738/0 (socket says 172.21.15.57:48738) 2024-04-25T23:54:02.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.724+0000 7fb755d9b640 1 -- 172.21.15.57:0/1912958689 learned_addr learned my addr 172.21.15.57:0/1912958689 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:54:02.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.724+0000 7fb755d9b640 1 -- 172.21.15.57:0/1912958689 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fb76013f500 msgr2=0x7fb760135ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-04-25T23:54:02.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.724+0000 7fb755d9b640 1 -- 172.21.15.57:0/1912958689 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fb76013f500 msgr2=0x7fb760135ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T23:54:02.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.724+0000 7fb755d9b640 1 --2- 172.21.15.57:0/1912958689 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fb76013f500 0x7fb760135ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T23:54:02.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.724+0000 7fb755d9b640 1 --2- 172.21.15.57:0/1912958689 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fb76013f500 0x7fb760135ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T23:54:02.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.724+0000 7fb75ed9d640 1 --2- 172.21.15.57:0/1912958689 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb760140340 0x7fb760136d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:02.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.724+0000 7fb75ed9d640 1 -- 172.21.15.57:0/1912958689 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fb76013f500 msgr2=0x7fb760135ff0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T23:54:02.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.724+0000 7fb75ed9d640 1 --2- 172.21.15.57:0/1912958689 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fb76013f500 0x7fb760135ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:02.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.724+0000 7fb75ed9d640 1 -- 172.21.15.57:0/1912958689 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb760071d00 msgr2=0x7fb7601da130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:02.711 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.724+0000 7fb75ed9d640 1 --2- 172.21.15.57:0/1912958689 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb760071d00 0x7fb7601da130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:02.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.724+0000 7fb75ed9d640 1 -- 172.21.15.57:0/1912958689 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb760136530 con 0x7fb760140340 2024-04-25T23:54:02.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.724+0000 7fb75ed9d640 1 --2- 172.21.15.57:0/1912958689 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb760140340 0x7fb760136d00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb758007d90 tx=0x7fb75800e4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:54:02.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.728+0000 7fb7577fe640 1 -- 172.21.15.57:0/1912958689 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75800edd0 con 0x7fb760140340 2024-04-25T23:54:02.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.728+0000 7fb764b0c640 1 -- 172.21.15.57:0/1912958689 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7601367e0 con 0x7fb760140340 2024-04-25T23:54:02.713 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.728+0000 7fb7577fe640 1 -- 172.21.15.57:0/1912958689 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7580039a0 con 0x7fb760140340 2024-04-25T23:54:02.713 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.728+0000 7fb7577fe640 1 -- 172.21.15.57:0/1912958689 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb758003d90 con 0x7fb760140340 2024-04-25T23:54:02.713 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.728+0000 7fb764b0c640 1 -- 172.21.15.57:0/1912958689 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb760133590 con 0x7fb760140340 2024-04-25T23:54:02.713 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.728+0000 7fb764b0c640 1 -- 172.21.15.57:0/1912958689 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7600761e0 con 0x7fb760140340 2024-04-25T23:54:02.713 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.728+0000 7fb7577fe640 1 -- 172.21.15.57:0/1912958689 <== mon.1 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 87410+0+0 (secure 0 0 0) 0x7fb75801b020 con 0x7fb760140340 2024-04-25T23:54:02.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.728+0000 7fb7577fe640 1 --2- 172.21.15.57:0/1912958689 >> v2:172.21.15.80:6816/2493457677 conn(0x7fb740067880 0x7fb740069d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:02.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.728+0000 7fb7577fe640 1 -- 172.21.15.57:0/1912958689 <== mon.1 v2:172.21.15.80:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 4525+0+0 (secure 0 0 0) 0x7fb758086840 con 0x7fb760140340 2024-04-25T23:54:02.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.728+0000 7fb75e59c640 1 --2- 172.21.15.57:0/1912958689 >> v2:172.21.15.80:6816/2493457677 conn(0x7fb740067880 0x7fb740069d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:02.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.732+0000 7fb75e59c640 1 --2- 172.21.15.57:0/1912958689 >> v2:172.21.15.80:6816/2493457677 conn(0x7fb740067880 0x7fb740069d40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb7500b1000 tx=0x7fb750004740 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:54:02.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.744+0000 7fb7577fe640 1 -- 172.21.15.57:0/1912958689 <== mon.1 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb7580578b0 con 0x7fb760140340 2024-04-25T23:54:02.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.908+0000 7fb764b0c640 1 -- 172.21.15.57:0/1912958689 --> v2:172.21.15.80:6816/2493457677 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7fb7601dae40 con 0x7fb740067880 2024-04-25T23:54:02.894 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.908+0000 7fb7577fe640 1 -- 172.21.15.57:0/1912958689 <== mgr.4100 v2:172.21.15.80:6816/2493457677 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fb7500a7600 con 0x7fb740067880 2024-04-25T23:54:02.895 INFO:teuthology.orchestra.run.smithi057.stderr:instructed osd(s) 2 to deep-scrub 2024-04-25T23:54:02.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.912+0000 7fb764b0c640 1 -- 172.21.15.57:0/1912958689 >> v2:172.21.15.80:6816/2493457677 conn(0x7fb740067880 msgr2=0x7fb740069d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:02.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.912+0000 7fb764b0c640 1 --2- 172.21.15.57:0/1912958689 >> v2:172.21.15.80:6816/2493457677 conn(0x7fb740067880 0x7fb740069d40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb7500b1000 tx=0x7fb750004740 comp rx=0 tx=0).stop 2024-04-25T23:54:02.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.912+0000 7fb764b0c640 1 -- 172.21.15.57:0/1912958689 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb760140340 msgr2=0x7fb760136d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:02.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.912+0000 7fb764b0c640 1 --2- 172.21.15.57:0/1912958689 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb760140340 0x7fb760136d00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb758007d90 tx=0x7fb75800e4b0 comp rx=0 tx=0).stop 2024-04-25T23:54:02.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.912+0000 7fb764b0c640 1 -- 172.21.15.57:0/1912958689 shutdown_connections 2024-04-25T23:54:02.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.912+0000 7fb764b0c640 1 --2- 172.21.15.57:0/1912958689 >> v2:172.21.15.80:6816/2493457677 conn(0x7fb740067880 0x7fb740069d40 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:02.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.912+0000 7fb764b0c640 1 --2- 172.21.15.57:0/1912958689 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb760140340 0x7fb760136d00 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:02.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.912+0000 7fb764b0c640 1 --2- 172.21.15.57:0/1912958689 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fb76013f500 0x7fb760135ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:02.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.912+0000 7fb764b0c640 1 --2- 172.21.15.57:0/1912958689 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb760071d00 0x7fb7601da130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:02.899 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.912+0000 7fb764b0c640 1 -- 172.21.15.57:0/1912958689 >> 172.21.15.57:0/1912958689 conn(0x7fb7600b5a70 msgr2=0x7fb760066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:54:02.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.912+0000 7fb764b0c640 1 -- 172.21.15.57:0/1912958689 shutdown_connections 2024-04-25T23:54:02.900 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:02.912+0000 7fb764b0c640 1 -- 172.21.15.57:0/1912958689 wait complete. 2024-04-25T23:54:02.917 INFO:tasks.ceph:Scrubbing osd.3 2024-04-25T23:54:02.918 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-25T23:54:03.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.052+0000 7efdd483d640 1 Processor -- start 2024-04-25T23:54:03.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.052+0000 7efdd483d640 1 -- start start 2024-04-25T23:54:03.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.052+0000 7efdd483d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7efdd0071ff0 0x7efdd0070310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:03.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.052+0000 7efdd483d640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7efdd00b0e40 0x7efdd013f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:03.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.052+0000 7efdd483d640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdd00723d0 con 0x7efdd013fd00 2024-04-25T23:54:03.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.052+0000 7efdd483d640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7efdd0070850 con 0x7efdd0071ff0 2024-04-25T23:54:03.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.052+0000 7efdd483d640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7efdd013f000 con 0x7efdd00b0e40 2024-04-25T23:54:03.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.052+0000 7efdcdd9b640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7efdd00b0e40 0x7efdd013f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:03.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.052+0000 7efdcdd9b640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7efdd00b0e40 0x7efdd013f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3301/0 says I am v2:172.21.15.57:48742/0 (socket says 172.21.15.57:48742) 2024-04-25T23:54:03.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.052+0000 7efdcdd9b640 1 -- 172.21.15.57:0/4142148673 learned_addr learned my addr 172.21.15.57:0/4142148673 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:54:03.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.052+0000 7efdce59c640 1 --2- 172.21.15.57:0/4142148673 >> v2:172.21.15.80:3300/0 conn(0x7efdd0071ff0 0x7efdd0070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:03.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.056+0000 7efdcdd9b640 1 -- 172.21.15.57:0/4142148673 >> v2:172.21.15.80:3300/0 conn(0x7efdd0071ff0 msgr2=0x7efdd0070310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:54:03.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.056+0000 7efdcdd9b640 1 --2- 172.21.15.57:0/4142148673 >> v2:172.21.15.80:3300/0 conn(0x7efdd0071ff0 0x7efdd0070310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:03.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.056+0000 7efdcdd9b640 1 -- 172.21.15.57:0/4142148673 >> v1:172.21.15.80:6789/0 conn(0x7efdd013fd00 legacy=0x7efdd013c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:03.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.056+0000 7efdcdd9b640 1 -- 172.21.15.57:0/4142148673 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdd00a9df0 con 0x7efdd00b0e40 2024-04-25T23:54:03.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.056+0000 7efdce59c640 1 --2- 172.21.15.57:0/4142148673 >> v2:172.21.15.80:3300/0 conn(0x7efdd0071ff0 0x7efdd0070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T23:54:03.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.056+0000 7efdcdd9b640 1 --2- 172.21.15.57:0/4142148673 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7efdd00b0e40 0x7efdd013f7c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7efdb8009870 tx=0x7efdb801c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ac49a4dd7cceed8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:54:03.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.056+0000 7efdcd59a640 1 -- 172.21.15.57:0/4142148673 <== mon.2 v2:172.21.15.57:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb80ab620 con 0x7efdd00b0e40 2024-04-25T23:54:03.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.056+0000 7efdcd59a640 1 -- 172.21.15.57:0/4142148673 <== mon.2 v2:172.21.15.57:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdb80b0070 con 0x7efdd00b0e40 2024-04-25T23:54:03.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.056+0000 7efdcd59a640 1 -- 172.21.15.57:0/4142148673 <== mon.2 v2:172.21.15.57:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb80ab9e0 con 0x7efdd00b0e40 2024-04-25T23:54:03.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.056+0000 7efdd483d640 1 -- 172.21.15.57:0/4142148673 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7efdd00b0e40 msgr2=0x7efdd013f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:03.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.056+0000 7efdd483d640 1 --2- 172.21.15.57:0/4142148673 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7efdd00b0e40 0x7efdd013f7c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7efdb8009870 tx=0x7efdb801c480 comp rx=0 tx=0).stop 2024-04-25T23:54:03.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.056+0000 7efdd483d640 1 -- 172.21.15.57:0/4142148673 shutdown_connections 2024-04-25T23:54:03.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.056+0000 7efdd483d640 1 --2- 172.21.15.57:0/4142148673 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7efdd00b0e40 0x7efdd013f7c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:03.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.056+0000 7efdd483d640 1 --2- 172.21.15.57:0/4142148673 >> v2:172.21.15.80:3300/0 conn(0x7efdd0071ff0 0x7efdd0070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:03.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.056+0000 7efdd483d640 1 -- 172.21.15.57:0/4142148673 >> 172.21.15.57:0/4142148673 conn(0x7efdd00b1390 msgr2=0x7efdd00b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:54:03.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.056+0000 7efdd483d640 1 -- 172.21.15.57:0/4142148673 shutdown_connections 2024-04-25T23:54:03.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.056+0000 7efdd483d640 1 -- 172.21.15.57:0/4142148673 wait complete. 2024-04-25T23:54:03.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.060+0000 7efdd483d640 1 Processor -- start 2024-04-25T23:54:03.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.060+0000 7efdd483d640 1 -- start start 2024-04-25T23:54:03.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.060+0000 7efdd483d640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7efdd0071ff0 0x7efdd013aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:03.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.060+0000 7efdd483d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efdd00b0e40 0x7efdd01dc570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:03.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.060+0000 7efdd483d640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efdd013fd00 0x7efdd00ae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:03.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.060+0000 7efdd483d640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7efdd0149d90 con 0x7efdd013fd00 2024-04-25T23:54:03.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.060+0000 7efdd483d640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7efdd0149c10 con 0x7efdd00b0e40 2024-04-25T23:54:03.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.060+0000 7efdd483d640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7efdd0149f10 con 0x7efdd0071ff0 2024-04-25T23:54:03.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.060+0000 7efdce59c640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7efdd0071ff0 0x7efdd013aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:03.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.060+0000 7efdce59c640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7efdd0071ff0 0x7efdd013aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3301/0 says I am v2:172.21.15.57:48748/0 (socket says 172.21.15.57:48748) 2024-04-25T23:54:03.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.060+0000 7efdce59c640 1 -- 172.21.15.57:0/2856659602 learned_addr learned my addr 172.21.15.57:0/2856659602 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:54:03.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.060+0000 7efdced9d640 1 --2- 172.21.15.57:0/2856659602 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efdd013fd00 0x7efdd00ae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:03.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.060+0000 7efdcdd9b640 1 --2- 172.21.15.57:0/2856659602 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efdd00b0e40 0x7efdd01dc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:03.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.060+0000 7efdce59c640 1 -- 172.21.15.57:0/2856659602 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efdd00b0e40 msgr2=0x7efdd01dc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:03.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.060+0000 7efdce59c640 1 --2- 172.21.15.57:0/2856659602 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efdd00b0e40 0x7efdd01dc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:03.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.060+0000 7efdce59c640 1 -- 172.21.15.57:0/2856659602 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efdd013fd00 msgr2=0x7efdd00ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:03.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.060+0000 7efdce59c640 1 --2- 172.21.15.57:0/2856659602 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efdd013fd00 0x7efdd00ae6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:03.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.060+0000 7efdce59c640 1 -- 172.21.15.57:0/2856659602 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdd00afb40 con 0x7efdd0071ff0 2024-04-25T23:54:03.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.060+0000 7efdce59c640 1 --2- 172.21.15.57:0/2856659602 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7efdd0071ff0 0x7efdd013aaf0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7efdc8009fc0 tx=0x7efdc800d560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:54:03.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.060+0000 7efdc6ffd640 1 -- 172.21.15.57:0/2856659602 <== mon.2 v2:172.21.15.57:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc801a070 con 0x7efdd0071ff0 2024-04-25T23:54:03.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.060+0000 7efdc6ffd640 1 -- 172.21.15.57:0/2856659602 <== mon.2 v2:172.21.15.57:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdc8003ba0 con 0x7efdd0071ff0 2024-04-25T23:54:03.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.060+0000 7efdd483d640 1 -- 172.21.15.57:0/2856659602 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdd00aff60 con 0x7efdd0071ff0 2024-04-25T23:54:03.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.060+0000 7efdc6ffd640 1 -- 172.21.15.57:0/2856659602 <== mon.2 v2:172.21.15.57:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc8004030 con 0x7efdd0071ff0 2024-04-25T23:54:03.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.060+0000 7efdd483d640 1 -- 172.21.15.57:0/2856659602 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7efdd01496d0 con 0x7efdd0071ff0 2024-04-25T23:54:03.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.060+0000 7efdd483d640 1 -- 172.21.15.57:0/2856659602 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efdd00761e0 con 0x7efdd0071ff0 2024-04-25T23:54:03.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.060+0000 7efdc6ffd640 1 -- 172.21.15.57:0/2856659602 <== mon.2 v2:172.21.15.57:3301/0 4 ==== mgrmap(e 5) ==== 87410+0+0 (secure 0 0 0) 0x7efdc801d070 con 0x7efdd0071ff0 2024-04-25T23:54:03.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.060+0000 7efdc6ffd640 1 --2- 172.21.15.57:0/2856659602 >> v2:172.21.15.80:6816/2493457677 conn(0x7efdb0067880 0x7efdb0069d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:03.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.064+0000 7efdc6ffd640 1 -- 172.21.15.57:0/2856659602 <== mon.2 v2:172.21.15.57:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 4525+0+0 (secure 0 0 0) 0x7efdc8086440 con 0x7efdd0071ff0 2024-04-25T23:54:03.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.064+0000 7efdc6ffd640 1 --2- 172.21.15.57:0/2856659602 >> v2:172.21.15.57:6800/1899803014 conn(0x7efdb006e5e0 0x7efdb0070a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:03.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.064+0000 7efdc6ffd640 1 -- 172.21.15.57:0/2856659602 --> v2:172.21.15.57:6800/1899803014 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efdc8011e50 con 0x7efdb006e5e0 2024-04-25T23:54:03.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.064+0000 7efdc6ffd640 1 -- 172.21.15.57:0/2856659602 <== mon.2 v2:172.21.15.57:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7efdc80868d0 con 0x7efdd0071ff0 2024-04-25T23:54:03.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.064+0000 7efdced9d640 1 --2- 172.21.15.57:0/2856659602 >> v2:172.21.15.57:6800/1899803014 conn(0x7efdb006e5e0 0x7efdb0070a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:03.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.064+0000 7efdcdd9b640 1 --2- 172.21.15.57:0/2856659602 >> v2:172.21.15.80:6816/2493457677 conn(0x7efdb0067880 0x7efdb0069d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:03.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.064+0000 7efdced9d640 1 --2- 172.21.15.57:0/2856659602 >> v2:172.21.15.57:6800/1899803014 conn(0x7efdb006e5e0 0x7efdb0070a00 crc :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:54:03.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.064+0000 7efdcdd9b640 1 --2- 172.21.15.57:0/2856659602 >> v2:172.21.15.80:6816/2493457677 conn(0x7efdb0067880 0x7efdb0069d40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7efdb80af000 tx=0x7efdb80a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:54:03.049 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.064+0000 7efdc6ffd640 1 -- 172.21.15.57:0/2856659602 <== osd.3 v2:172.21.15.57:6800/1899803014 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efdc8011e50 con 0x7efdb006e5e0 2024-04-25T23:54:03.061 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.076+0000 7efdd483d640 1 -- 172.21.15.57:0/2856659602 --> v2:172.21.15.57:6800/1899803014 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7efdd0139e70 con 0x7efdb006e5e0 2024-04-25T23:54:03.062 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.076+0000 7efdc6ffd640 1 -- 172.21.15.57:0/2856659602 <== osd.3 v2:172.21.15.57:6800/1899803014 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7efdd0139e70 con 0x7efdb006e5e0 2024-04-25T23:54:03.062 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-25T23:54:03.062 INFO:teuthology.orchestra.run.smithi057.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T23:54:03.062 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-25T23:54:03.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.076+0000 7efdd483d640 1 -- 172.21.15.57:0/2856659602 >> v2:172.21.15.57:6800/1899803014 conn(0x7efdb006e5e0 msgr2=0x7efdb0070a00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:03.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.076+0000 7efdd483d640 1 --2- 172.21.15.57:0/2856659602 >> v2:172.21.15.57:6800/1899803014 conn(0x7efdb006e5e0 0x7efdb0070a00 crc :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:03.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.076+0000 7efdd483d640 1 -- 172.21.15.57:0/2856659602 >> v2:172.21.15.80:6816/2493457677 conn(0x7efdb0067880 msgr2=0x7efdb0069d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:03.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.076+0000 7efdd483d640 1 --2- 172.21.15.57:0/2856659602 >> v2:172.21.15.80:6816/2493457677 conn(0x7efdb0067880 0x7efdb0069d40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7efdb80af000 tx=0x7efdb80a7000 comp rx=0 tx=0).stop 2024-04-25T23:54:03.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.076+0000 7efdd483d640 1 -- 172.21.15.57:0/2856659602 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7efdd0071ff0 msgr2=0x7efdd013aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:03.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.076+0000 7efdd483d640 1 --2- 172.21.15.57:0/2856659602 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7efdd0071ff0 0x7efdd013aaf0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7efdc8009fc0 tx=0x7efdc800d560 comp rx=0 tx=0).stop 2024-04-25T23:54:03.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.076+0000 7efdced9d640 1 -- 172.21.15.57:0/2856659602 reap_dead start 2024-04-25T23:54:03.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.076+0000 7efdd483d640 1 -- 172.21.15.57:0/2856659602 shutdown_connections 2024-04-25T23:54:03.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.076+0000 7efdd483d640 1 -- 172.21.15.57:0/2856659602 >> 172.21.15.57:0/2856659602 conn(0x7efdd00b1390 msgr2=0x7efdd00709e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:54:03.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.076+0000 7efdd483d640 1 -- 172.21.15.57:0/2856659602 shutdown_connections 2024-04-25T23:54:03.064 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.076+0000 7efdd483d640 1 -- 172.21.15.57:0/2856659602 wait complete. 2024-04-25T23:54:03.079 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-25T23:54:03.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.212+0000 7f193df0d640 1 Processor -- start 2024-04-25T23:54:03.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.212+0000 7f193df0d640 1 -- start start 2024-04-25T23:54:03.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.212+0000 7f193df0d640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f1938071c00 0x7f1938146180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:03.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.212+0000 7f193df0d640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f1938072050 0x7f19381466c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:03.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.212+0000 7f193df0d640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f193806bea0 con 0x7f193806bac0 2024-04-25T23:54:03.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.212+0000 7f193df0d640 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 -- 0x7f19380b1910 con 0x7f1938071c00 2024-04-25T23:54:03.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.212+0000 7f193df0d640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f19380b1670 con 0x7f1938072050 2024-04-25T23:54:03.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.212+0000 7f192ffff640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f1938071c00 0x7f1938146180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:03.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.212+0000 7f192ffff640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f1938071c00 0x7f1938146180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37244/0 (socket says 172.21.15.57:37244) 2024-04-25T23:54:03.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.212+0000 7f192ffff640 1 -- 172.21.15.57:0/303673944 learned_addr learned my addr 172.21.15.57:0/303673944 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:54:03.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.212+0000 7f1937fff640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f1938072050 0x7f19381466c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:03.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.212+0000 7f192ffff640 1 -- 172.21.15.57:0/303673944 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f1938072050 msgr2=0x7f19381466c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:54:03.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.212+0000 7f192ffff640 1 --2- 172.21.15.57:0/303673944 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f1938072050 0x7f19381466c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:03.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.212+0000 7f192ffff640 1 -- 172.21.15.57:0/303673944 >> v1:172.21.15.80:6789/0 conn(0x7f193806bac0 legacy=0x7f1938141fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:03.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.212+0000 7f192ffff640 1 -- 172.21.15.57:0/303673944 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1938072550 con 0x7f1938071c00 2024-04-25T23:54:03.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.212+0000 7f1937fff640 1 --2- 172.21.15.57:0/303673944 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f1938072050 0x7f19381466c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T23:54:03.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.216+0000 7f192ffff640 1 --2- 172.21.15.57:0/303673944 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1938071c00 0x7f1938146180 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1928009870 tx=0x7f192801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37987bf819847a03 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:54:03.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.216+0000 7f1936ffd640 1 -- 172.21.15.57:0/303673944 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19280aa610 con 0x7f1938071c00 2024-04-25T23:54:03.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.216+0000 7f1936ffd640 1 -- 172.21.15.57:0/303673944 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19280af070 con 0x7f1938071c00 2024-04-25T23:54:03.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.216+0000 7f1936ffd640 1 -- 172.21.15.57:0/303673944 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19280aa9d0 con 0x7f1938071c00 2024-04-25T23:54:03.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.216+0000 7f193df0d640 1 -- 172.21.15.57:0/303673944 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1938071c00 msgr2=0x7f1938146180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:03.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.216+0000 7f193df0d640 1 --2- 172.21.15.57:0/303673944 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1938071c00 0x7f1938146180 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1928009870 tx=0x7f192801c490 comp rx=0 tx=0).stop 2024-04-25T23:54:03.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.216+0000 7f193df0d640 1 -- 172.21.15.57:0/303673944 shutdown_connections 2024-04-25T23:54:03.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.216+0000 7f193df0d640 1 --2- 172.21.15.57:0/303673944 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f1938072050 0x7f19381466c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:03.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.216+0000 7f193df0d640 1 --2- 172.21.15.57:0/303673944 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1938071c00 0x7f1938146180 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:03.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.216+0000 7f193df0d640 1 -- 172.21.15.57:0/303673944 >> 172.21.15.57:0/303673944 conn(0x7f19380b5420 msgr2=0x7f19380b5830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:54:03.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.216+0000 7f193df0d640 1 -- 172.21.15.57:0/303673944 shutdown_connections 2024-04-25T23:54:03.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.216+0000 7f193df0d640 1 -- 172.21.15.57:0/303673944 wait complete. 2024-04-25T23:54:03.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.216+0000 7f193df0d640 1 Processor -- start 2024-04-25T23:54:03.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.216+0000 7f193df0d640 1 -- start start 2024-04-25T23:54:03.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.220+0000 7f193df0d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f193806bac0 0x7f19381def50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:03.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.220+0000 7f193df0d640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1938071c00 0x7f19381df5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:03.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.220+0000 7f193df0d640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f1938072050 0x7f193813ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:03.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.220+0000 7f193df0d640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f1938150120 con 0x7f1938071c00 2024-04-25T23:54:03.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.220+0000 7f193df0d640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f193814ffa0 con 0x7f193806bac0 2024-04-25T23:54:03.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.220+0000 7f193df0d640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f19381502a0 con 0x7f1938072050 2024-04-25T23:54:03.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.220+0000 7f192ffff640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1938071c00 0x7f19381df5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:03.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.220+0000 7f1937fff640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f1938072050 0x7f193813ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:03.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.220+0000 7f192ffff640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1938071c00 0x7f19381df5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37260/0 (socket says 172.21.15.57:37260) 2024-04-25T23:54:03.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.220+0000 7f1937fff640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f1938072050 0x7f193813ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3301/0 says I am v2:172.21.15.57:48768/0 (socket says 172.21.15.57:48768) 2024-04-25T23:54:03.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.220+0000 7f192ffff640 1 -- 172.21.15.57:0/3789717108 learned_addr learned my addr 172.21.15.57:0/3789717108 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:54:03.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.220+0000 7f19377fe640 1 --2- 172.21.15.57:0/3789717108 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f193806bac0 0x7f19381def50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:03.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.220+0000 7f192ffff640 1 -- 172.21.15.57:0/3789717108 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f1938072050 msgr2=0x7f193813ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:03.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.220+0000 7f192ffff640 1 --2- 172.21.15.57:0/3789717108 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f1938072050 0x7f193813ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:03.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.220+0000 7f192ffff640 1 -- 172.21.15.57:0/3789717108 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f193806bac0 msgr2=0x7f19381def50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:03.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.220+0000 7f192ffff640 1 --2- 172.21.15.57:0/3789717108 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f193806bac0 0x7f19381def50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:03.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.220+0000 7f192ffff640 1 -- 172.21.15.57:0/3789717108 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1938144080 con 0x7f1938071c00 2024-04-25T23:54:03.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.220+0000 7f1937fff640 1 --2- 172.21.15.57:0/3789717108 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f1938072050 0x7f193813ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T23:54:03.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.220+0000 7f19377fe640 1 --2- 172.21.15.57:0/3789717108 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f193806bac0 0x7f19381def50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T23:54:03.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.220+0000 7f192ffff640 1 --2- 172.21.15.57:0/3789717108 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1938071c00 0x7f19381df5c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f19280ac040 tx=0x7f192801acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:54:03.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.220+0000 7f1934ff9640 1 -- 172.21.15.57:0/3789717108 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19280bd5f0 con 0x7f1938071c00 2024-04-25T23:54:03.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.220+0000 7f1934ff9640 1 -- 172.21.15.57:0/3789717108 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19280bd7b0 con 0x7f1938071c00 2024-04-25T23:54:03.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.220+0000 7f193df0d640 1 -- 172.21.15.57:0/3789717108 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19381dfb00 con 0x7f1938071c00 2024-04-25T23:54:03.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.220+0000 7f1934ff9640 1 -- 172.21.15.57:0/3789717108 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19280bdba0 con 0x7f1938071c00 2024-04-25T23:54:03.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.220+0000 7f193df0d640 1 -- 172.21.15.57:0/3789717108 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f193813a520 con 0x7f1938071c00 2024-04-25T23:54:03.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.220+0000 7f193df0d640 1 -- 172.21.15.57:0/3789717108 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19380761e0 con 0x7f1938071c00 2024-04-25T23:54:03.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.220+0000 7f1934ff9640 1 -- 172.21.15.57:0/3789717108 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) ==== 87410+0+0 (secure 0 0 0) 0x7f19280b33f0 con 0x7f1938071c00 2024-04-25T23:54:03.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.220+0000 7f1934ff9640 1 --2- 172.21.15.57:0/3789717108 >> v2:172.21.15.80:6816/2493457677 conn(0x7f1918067880 0x7f1918069d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:03.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.220+0000 7f1934ff9640 1 -- 172.21.15.57:0/3789717108 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 4525+0+0 (secure 0 0 0) 0x7f1928120180 con 0x7f1938071c00 2024-04-25T23:54:03.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.220+0000 7f19377fe640 1 --2- 172.21.15.57:0/3789717108 >> v2:172.21.15.80:6816/2493457677 conn(0x7f1918067880 0x7f1918069d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:03.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.224+0000 7f19377fe640 1 --2- 172.21.15.57:0/3789717108 >> v2:172.21.15.80:6816/2493457677 conn(0x7f1918067880 0x7f1918069d40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1920000f70 tx=0x7f1920009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:54:03.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.224+0000 7f1934ff9640 1 -- 172.21.15.57:0/3789717108 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f19280a7070 con 0x7f1938071c00 2024-04-25T23:54:03.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.392+0000 7f193df0d640 1 -- 172.21.15.57:0/3789717108 --> v2:172.21.15.80:6816/2493457677 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f19380b5ba0 con 0x7f1918067880 2024-04-25T23:54:03.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.392+0000 7f1934ff9640 1 -- 172.21.15.57:0/3789717108 <== mgr.4100 v2:172.21.15.80:6816/2493457677 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f19380b5ba0 con 0x7f1918067880 2024-04-25T23:54:03.376 INFO:teuthology.orchestra.run.smithi057.stderr:instructed osd(s) 3 to deep-scrub 2024-04-25T23:54:03.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.392+0000 7f193df0d640 1 -- 172.21.15.57:0/3789717108 >> v2:172.21.15.80:6816/2493457677 conn(0x7f1918067880 msgr2=0x7f1918069d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:03.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.392+0000 7f193df0d640 1 --2- 172.21.15.57:0/3789717108 >> v2:172.21.15.80:6816/2493457677 conn(0x7f1918067880 0x7f1918069d40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1920000f70 tx=0x7f1920009040 comp rx=0 tx=0).stop 2024-04-25T23:54:03.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.392+0000 7f193df0d640 1 -- 172.21.15.57:0/3789717108 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1938071c00 msgr2=0x7f19381df5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:03.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.392+0000 7f193df0d640 1 --2- 172.21.15.57:0/3789717108 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1938071c00 0x7f19381df5c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f19280ac040 tx=0x7f192801acd0 comp rx=0 tx=0).stop 2024-04-25T23:54:03.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.392+0000 7f193df0d640 1 -- 172.21.15.57:0/3789717108 shutdown_connections 2024-04-25T23:54:03.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.392+0000 7f193df0d640 1 --2- 172.21.15.57:0/3789717108 >> v2:172.21.15.80:6816/2493457677 conn(0x7f1918067880 0x7f1918069d40 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:03.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.392+0000 7f193df0d640 1 --2- 172.21.15.57:0/3789717108 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f1938072050 0x7f193813ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:03.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.392+0000 7f193df0d640 1 --2- 172.21.15.57:0/3789717108 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1938071c00 0x7f19381df5c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:03.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.392+0000 7f193df0d640 1 --2- 172.21.15.57:0/3789717108 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f193806bac0 0x7f19381def50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:03.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.392+0000 7f193df0d640 1 -- 172.21.15.57:0/3789717108 >> 172.21.15.57:0/3789717108 conn(0x7f19380b5420 msgr2=0x7f19380b2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:54:03.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.392+0000 7f193df0d640 1 -- 172.21.15.57:0/3789717108 shutdown_connections 2024-04-25T23:54:03.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.392+0000 7f193df0d640 1 -- 172.21.15.57:0/3789717108 wait complete. 2024-04-25T23:54:03.396 INFO:tasks.ceph:Scrubbing osd.4 2024-04-25T23:54:03.396 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-25T23:54:03.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.536+0000 7fe82f3c2640 1 Processor -- start 2024-04-25T23:54:03.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.536+0000 7fe82f3c2640 1 -- start start 2024-04-25T23:54:03.524 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.540+0000 7fe82f3c2640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fe828146d00 0x7fe8280710e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:03.524 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.540+0000 7fe82f3c2640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe828074f30 con 0x7fe828074b50 2024-04-25T23:54:03.524 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.540+0000 7fe82f3c2640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe828072140 con 0x7fe828071cf0 2024-04-25T23:54:03.524 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.540+0000 7fe82f3c2640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7fe828146180 con 0x7fe828146d00 2024-04-25T23:54:03.524 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.540+0000 7fe82c95d640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fe828146d00 0x7fe8280710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:03.524 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.540+0000 7fe82d15e640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fe828074b50 0x7fe8281465f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:35394/0 (socket says 172.21.15.57:35394) 2024-04-25T23:54:03.524 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.540+0000 7fe82d15e640 1 -- 172.21.15.57:0/1807921243 learned_addr learned my addr 172.21.15.57:0/1807921243 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-04-25T23:54:03.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.540+0000 7fe82c95d640 1 -- 172.21.15.57:0/1807921243 >> v1:172.21.15.80:6789/0 conn(0x7fe828071cf0 legacy=0x7fe828141ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:54:03.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.540+0000 7fe82c95d640 1 -- 172.21.15.57:0/1807921243 >> v1:172.21.15.57:6789/0 conn(0x7fe828074b50 legacy=0x7fe8281465f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:03.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.540+0000 7fe81ffff640 1 -- 172.21.15.57:0/1807921243 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3193536229 0 0) 0x7fe824001120 con 0x7fe828074b50 2024-04-25T23:54:03.525 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.540+0000 7fe82c95d640 1 -- 172.21.15.57:0/1807921243 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe828142c00 con 0x7fe828146d00 2024-04-25T23:54:03.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.540+0000 7fe82c95d640 1 --2- 172.21.15.57:0/1807921243 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fe828146d00 0x7fe8280710e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe8180097f0 tx=0x7fe81801c3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35507c5798e94a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:54:03.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.540+0000 7fe81ffff640 1 -- 172.21.15.57:0/1807921243 <== mon.2 v2:172.21.15.57:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8180b0070 con 0x7fe828146d00 2024-04-25T23:54:03.527 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.544+0000 7fe81ffff640 1 -- 172.21.15.57:0/1807921243 <== mon.2 v2:172.21.15.57:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8180ab600 con 0x7fe828146d00 2024-04-25T23:54:03.527 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.544+0000 7fe81ffff640 1 -- 172.21.15.57:0/1807921243 <== mon.2 v2:172.21.15.57:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8180ab9d0 con 0x7fe828146d00 2024-04-25T23:54:03.527 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.544+0000 7fe82f3c2640 1 -- 172.21.15.57:0/1807921243 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fe828146d00 msgr2=0x7fe8280710e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:03.528 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.544+0000 7fe82f3c2640 1 --2- 172.21.15.57:0/1807921243 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fe828146d00 0x7fe8280710e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe8180097f0 tx=0x7fe81801c3f0 comp rx=0 tx=0).stop 2024-04-25T23:54:03.528 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.544+0000 7fe82f3c2640 1 -- 172.21.15.57:0/1807921243 shutdown_connections 2024-04-25T23:54:03.528 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.544+0000 7fe82f3c2640 1 --2- 172.21.15.57:0/1807921243 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fe828146d00 0x7fe8280710e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:03.528 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.544+0000 7fe82f3c2640 1 -- 172.21.15.57:0/1807921243 >> 172.21.15.57:0/1807921243 conn(0x7fe8280b5910 msgr2=0x7fe8280b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:54:03.528 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.544+0000 7fe82f3c2640 1 -- 172.21.15.57:0/1807921243 shutdown_connections 2024-04-25T23:54:03.528 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.544+0000 7fe82f3c2640 1 -- 172.21.15.57:0/1807921243 wait complete. 2024-04-25T23:54:03.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.544+0000 7fe82f3c2640 1 Processor -- start 2024-04-25T23:54:03.529 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.544+0000 7fe82f3c2640 1 -- start start 2024-04-25T23:54:03.530 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.544+0000 7fe82f3c2640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe828071cf0 0x7fe8280af550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:03.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.544+0000 7fe82f3c2640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe828074b50 0x7fe8280afb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:03.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.544+0000 7fe82f3c2640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fe828146d00 0x7fe8280b00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:03.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.544+0000 7fe82f3c2640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fe828151180 con 0x7fe828074b50 2024-04-25T23:54:03.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.544+0000 7fe82f3c2640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe828151000 con 0x7fe828071cf0 2024-04-25T23:54:03.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.544+0000 7fe82f3c2640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7fe828151300 con 0x7fe828146d00 2024-04-25T23:54:03.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.544+0000 7fe82d95f640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fe828146d00 0x7fe8280b00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:03.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.544+0000 7fe82d95f640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fe828146d00 0x7fe8280b00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3301/0 says I am v2:172.21.15.57:48794/0 (socket says 172.21.15.57:48794) 2024-04-25T23:54:03.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.544+0000 7fe82d95f640 1 -- 172.21.15.57:0/3740455784 learned_addr learned my addr 172.21.15.57:0/3740455784 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:54:03.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.544+0000 7fe82d15e640 1 --2- 172.21.15.57:0/3740455784 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe828071cf0 0x7fe8280af550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:03.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.544+0000 7fe82c95d640 1 --2- 172.21.15.57:0/3740455784 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe828074b50 0x7fe8280afb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:03.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.544+0000 7fe82d95f640 1 -- 172.21.15.57:0/3740455784 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe828071cf0 msgr2=0x7fe8280af550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:03.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.544+0000 7fe82d95f640 1 --2- 172.21.15.57:0/3740455784 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe828071cf0 0x7fe8280af550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:03.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.544+0000 7fe82d95f640 1 -- 172.21.15.57:0/3740455784 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe828074b50 msgr2=0x7fe8280afb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:03.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.544+0000 7fe82d95f640 1 --2- 172.21.15.57:0/3740455784 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe828074b50 0x7fe8280afb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:03.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.544+0000 7fe82d95f640 1 -- 172.21.15.57:0/3740455784 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8280b05e0 con 0x7fe828146d00 2024-04-25T23:54:03.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.544+0000 7fe82d95f640 1 --2- 172.21.15.57:0/3740455784 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fe828146d00 0x7fe8280b00a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe82000c7b0 tx=0x7fe82000cc70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:54:03.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.544+0000 7fe81dffb640 1 -- 172.21.15.57:0/3740455784 <== mon.2 v2:172.21.15.57:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe820004b70 con 0x7fe828146d00 2024-04-25T23:54:03.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.544+0000 7fe81dffb640 1 -- 172.21.15.57:0/3740455784 <== mon.2 v2:172.21.15.57:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe820004d10 con 0x7fe828146d00 2024-04-25T23:54:03.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.548+0000 7fe82f3c2640 1 -- 172.21.15.57:0/3740455784 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe82813f3d0 con 0x7fe828146d00 2024-04-25T23:54:03.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.548+0000 7fe81dffb640 1 -- 172.21.15.57:0/3740455784 <== mon.2 v2:172.21.15.57:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe82000aa90 con 0x7fe828146d00 2024-04-25T23:54:03.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.548+0000 7fe82f3c2640 1 -- 172.21.15.57:0/3740455784 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe828135e50 con 0x7fe828146d00 2024-04-25T23:54:03.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.548+0000 7fe82f3c2640 1 -- 172.21.15.57:0/3740455784 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe828138e90 con 0x7fe828146d00 2024-04-25T23:54:03.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.548+0000 7fe81dffb640 1 -- 172.21.15.57:0/3740455784 <== mon.2 v2:172.21.15.57:3301/0 4 ==== mgrmap(e 5) ==== 87410+0+0 (secure 0 0 0) 0x7fe82000ac30 con 0x7fe828146d00 2024-04-25T23:54:03.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.548+0000 7fe81dffb640 1 --2- 172.21.15.57:0/3740455784 >> v2:172.21.15.80:6816/2493457677 conn(0x7fe80c067880 0x7fe80c069d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:03.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.548+0000 7fe81dffb640 1 -- 172.21.15.57:0/3740455784 <== mon.2 v2:172.21.15.57:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 4525+0+0 (secure 0 0 0) 0x7fe8200858a0 con 0x7fe828146d00 2024-04-25T23:54:03.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.548+0000 7fe81dffb640 1 --2- 172.21.15.57:0/3740455784 >> v2:172.21.15.80:6800/4069821322 conn(0x7fe80c06e5e0 0x7fe80c070a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:03.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.548+0000 7fe81dffb640 1 -- 172.21.15.57:0/3740455784 --> v2:172.21.15.80:6800/4069821322 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe820007a00 con 0x7fe80c06e5e0 2024-04-25T23:54:03.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.548+0000 7fe81dffb640 1 -- 172.21.15.57:0/3740455784 <== mon.2 v2:172.21.15.57:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fe82008a2c0 con 0x7fe828146d00 2024-04-25T23:54:03.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.548+0000 7fe82d15e640 1 --2- 172.21.15.57:0/3740455784 >> v2:172.21.15.80:6816/2493457677 conn(0x7fe80c067880 0x7fe80c069d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:03.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.548+0000 7fe82c95d640 1 --2- 172.21.15.57:0/3740455784 >> v2:172.21.15.80:6800/4069821322 conn(0x7fe80c06e5e0 0x7fe80c070a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:03.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.548+0000 7fe82d15e640 1 --2- 172.21.15.57:0/3740455784 >> v2:172.21.15.80:6816/2493457677 conn(0x7fe80c067880 0x7fe80c069d40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe824000f70 tx=0x7fe824007b60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:54:03.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.548+0000 7fe82c95d640 1 -- 172.21.15.57:0/3740455784 >> v2:172.21.15.80:6800/4069821322 conn(0x7fe80c06e5e0 msgr2=0x7fe80c070a00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-04-25T23:54:03.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.548+0000 7fe82c95d640 1 -- 172.21.15.57:0/3740455784 >> v2:172.21.15.80:6800/4069821322 conn(0x7fe80c06e5e0 msgr2=0x7fe80c070a00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T23:54:03.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.548+0000 7fe82c95d640 1 --2- 172.21.15.57:0/3740455784 >> v2:172.21.15.80:6800/4069821322 conn(0x7fe80c06e5e0 0x7fe80c070a00 crc :-1 s=COMPRESSION_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T23:54:03.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.548+0000 7fe82c95d640 1 --2- 172.21.15.57:0/3740455784 >> v2:172.21.15.80:6800/4069821322 conn(0x7fe80c06e5e0 0x7fe80c070a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T23:54:03.735 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.752+0000 7fe82c95d640 1 --2- 172.21.15.57:0/3740455784 >> v2:172.21.15.80:6800/4069821322 conn(0x7fe80c06e5e0 0x7fe80c070a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:03.736 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.752+0000 7fe82c95d640 1 --2- 172.21.15.57:0/3740455784 >> v2:172.21.15.80:6800/4069821322 conn(0x7fe80c06e5e0 0x7fe80c070a00 crc :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:54:03.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.752+0000 7fe81dffb640 1 -- 172.21.15.57:0/3740455784 <== osd.4 v2:172.21.15.80:6800/4069821322 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe820007a00 con 0x7fe80c06e5e0 2024-04-25T23:54:03.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.780+0000 7fe82f3c2640 1 -- 172.21.15.57:0/3740455784 --> v2:172.21.15.80:6800/4069821322 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fe828139270 con 0x7fe80c06e5e0 2024-04-25T23:54:03.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.780+0000 7fe81dffb640 1 -- 172.21.15.57:0/3740455784 <== osd.4 v2:172.21.15.80:6800/4069821322 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fe828139270 con 0x7fe80c06e5e0 2024-04-25T23:54:03.765 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-25T23:54:03.765 INFO:teuthology.orchestra.run.smithi057.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T23:54:03.765 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-25T23:54:03.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.780+0000 7fe82f3c2640 1 -- 172.21.15.57:0/3740455784 >> v2:172.21.15.80:6800/4069821322 conn(0x7fe80c06e5e0 msgr2=0x7fe80c070a00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:03.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.780+0000 7fe82f3c2640 1 --2- 172.21.15.57:0/3740455784 >> v2:172.21.15.80:6800/4069821322 conn(0x7fe80c06e5e0 0x7fe80c070a00 crc :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:03.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.780+0000 7fe82f3c2640 1 -- 172.21.15.57:0/3740455784 >> v2:172.21.15.80:6816/2493457677 conn(0x7fe80c067880 msgr2=0x7fe80c069d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:03.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.780+0000 7fe82f3c2640 1 --2- 172.21.15.57:0/3740455784 >> v2:172.21.15.80:6816/2493457677 conn(0x7fe80c067880 0x7fe80c069d40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe824000f70 tx=0x7fe824007b60 comp rx=0 tx=0).stop 2024-04-25T23:54:03.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.780+0000 7fe82f3c2640 1 -- 172.21.15.57:0/3740455784 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fe828146d00 msgr2=0x7fe8280b00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:03.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.780+0000 7fe82f3c2640 1 --2- 172.21.15.57:0/3740455784 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fe828146d00 0x7fe8280b00a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe82000c7b0 tx=0x7fe82000cc70 comp rx=0 tx=0).stop 2024-04-25T23:54:03.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.780+0000 7fe82d95f640 1 -- 172.21.15.57:0/3740455784 reap_dead start 2024-04-25T23:54:03.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.780+0000 7fe82f3c2640 1 -- 172.21.15.57:0/3740455784 shutdown_connections 2024-04-25T23:54:03.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.780+0000 7fe82f3c2640 1 -- 172.21.15.57:0/3740455784 >> 172.21.15.57:0/3740455784 conn(0x7fe8280b5910 msgr2=0x7fe8280b3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:54:03.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.780+0000 7fe82f3c2640 1 -- 172.21.15.57:0/3740455784 shutdown_connections 2024-04-25T23:54:03.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.780+0000 7fe82f3c2640 1 -- 172.21.15.57:0/3740455784 wait complete. 2024-04-25T23:54:03.786 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-25T23:54:03.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.932+0000 7fe0cf1c4640 1 Processor -- start 2024-04-25T23:54:03.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.932+0000 7fe0cf1c4640 1 -- start start 2024-04-25T23:54:03.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.932+0000 7fe0cf1c4640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe0c80a9b30 0x7fe0c814c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:03.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.932+0000 7fe0cf1c4640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0c8072420 con 0x7fe0c8072040 2024-04-25T23:54:03.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.932+0000 7fe0cf1c4640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0c80abaa0 con 0x7fe0c8140bf0 2024-04-25T23:54:03.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.932+0000 7fe0cf1c4640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fe0c81440f0 con 0x7fe0c80a9b30 2024-04-25T23:54:03.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.932+0000 7fe0ccf60640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fe0c8072040 0x7fe0c813ffd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:35396/0 (socket says 172.21.15.57:35396) 2024-04-25T23:54:03.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.932+0000 7fe0ccf60640 1 -- 172.21.15.57:0/3028051924 learned_addr learned my addr 172.21.15.57:0/3028051924 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-04-25T23:54:03.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.932+0000 7fe0cd761640 1 --2- 172.21.15.57:0/3028051924 >> v2:172.21.15.80:3300/0 conn(0x7fe0c80a9b30 0x7fe0c814c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:03.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.932+0000 7fe0c77fe640 1 -- 172.21.15.57:0/3028051924 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2973990665 0 0) 0x7fe0bc001120 con 0x7fe0c8072040 2024-04-25T23:54:03.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.932+0000 7fe0c77fe640 1 -- 172.21.15.57:0/3028051924 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe0b8003530 con 0x7fe0c8072040 2024-04-25T23:54:03.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.936+0000 7fe0c77fe640 1 -- 172.21.15.57:0/3028051924 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3575908985 0 0) 0x7fe0b8003530 con 0x7fe0c8072040 2024-04-25T23:54:03.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.936+0000 7fe0c77fe640 1 -- 172.21.15.57:0/3028051924 >> v2:172.21.15.80:3300/0 conn(0x7fe0c80a9b30 msgr2=0x7fe0c814c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:54:03.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.936+0000 7fe0c77fe640 1 --2- 172.21.15.57:0/3028051924 >> v2:172.21.15.80:3300/0 conn(0x7fe0c80a9b30 0x7fe0c814c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:03.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.936+0000 7fe0c77fe640 1 -- 172.21.15.57:0/3028051924 >> v1:172.21.15.80:6789/0 conn(0x7fe0c8140bf0 legacy=0x7fe0c8144580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:03.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.936+0000 7fe0c77fe640 1 -- 172.21.15.57:0/3028051924 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0c80a96e0 con 0x7fe0c8072040 2024-04-25T23:54:03.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.936+0000 7fe0c77fe640 1 -- 172.21.15.57:0/3028051924 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2627952221 0 0) 0x7fe0bc001850 con 0x7fe0c8072040 2024-04-25T23:54:03.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.936+0000 7fe0c77fe640 1 -- 172.21.15.57:0/3028051924 >> v1:172.21.15.57:6789/0 conn(0x7fe0c8072040 legacy=0x7fe0c813ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:03.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.936+0000 7fe0c77fe640 1 --2- 172.21.15.57:0/3028051924 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0b80039c0 0x7fe0b8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:03.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.936+0000 7fe0c77fe640 1 --2- 172.21.15.57:0/3028051924 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fe0b8004810 0x7fe0b8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:03.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.936+0000 7fe0c77fe640 1 --2- 172.21.15.57:0/3028051924 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe0b80052f0 0x7fe0b800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:03.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.936+0000 7fe0c77fe640 1 -- 172.21.15.57:0/3028051924 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c81440f0 con 0x7fe0b80052f0 2024-04-25T23:54:03.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.936+0000 7fe0c77fe640 1 -- 172.21.15.57:0/3028051924 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b800c1c0 con 0x7fe0b80039c0 2024-04-25T23:54:03.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.936+0000 7fe0c77fe640 1 -- 172.21.15.57:0/3028051924 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7fe0b800c3c0 con 0x7fe0b8004810 2024-04-25T23:54:03.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.936+0000 7fe0c7fff640 1 --2- 172.21.15.57:0/3028051924 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fe0b8004810 0x7fe0b8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:03.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.936+0000 7fe0cd761640 1 --2- 172.21.15.57:0/3028051924 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe0b80052f0 0x7fe0b800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:03.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.936+0000 7fe0ccf60640 1 --2- 172.21.15.57:0/3028051924 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0b80039c0 0x7fe0b8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:03.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.936+0000 7fe0c7fff640 1 -- 172.21.15.57:0/3028051924 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0b80039c0 msgr2=0x7fe0b8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:54:03.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.936+0000 7fe0c7fff640 1 --2- 172.21.15.57:0/3028051924 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0b80039c0 0x7fe0b8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:03.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.936+0000 7fe0c7fff640 1 -- 172.21.15.57:0/3028051924 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe0b80052f0 msgr2=0x7fe0b800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:54:03.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.936+0000 7fe0c7fff640 1 --2- 172.21.15.57:0/3028051924 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe0b80052f0 0x7fe0b800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:03.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.936+0000 7fe0c7fff640 1 -- 172.21.15.57:0/3028051924 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0b8003530 con 0x7fe0b8004810 2024-04-25T23:54:03.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.936+0000 7fe0cd761640 1 --2- 172.21.15.57:0/3028051924 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe0b80052f0 0x7fe0b800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T23:54:03.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.936+0000 7fe0cd761640 1 -- 172.21.15.57:0/3028051924 reap_dead start 2024-04-25T23:54:03.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.940+0000 7fe0c7fff640 1 --2- 172.21.15.57:0/3028051924 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fe0b8004810 0x7fe0b8004c90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4001750 tx=0x7fe0b401c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4bfb241b7a4c835 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:54:03.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.940+0000 7fe0c77fe640 1 -- 172.21.15.57:0/3028051924 <== mon.2 v2:172.21.15.57:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b40013a0 con 0x7fe0b8004810 2024-04-25T23:54:03.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.940+0000 7fe0c77fe640 1 -- 172.21.15.57:0/3028051924 <== mon.2 v2:172.21.15.57:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0b40052f0 con 0x7fe0b8004810 2024-04-25T23:54:03.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.940+0000 7fe0c77fe640 1 -- 172.21.15.57:0/3028051924 <== mon.2 v2:172.21.15.57:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b40a4e60 con 0x7fe0b8004810 2024-04-25T23:54:03.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.940+0000 7fe0cf1c4640 1 -- 172.21.15.57:0/3028051924 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fe0b8004810 msgr2=0x7fe0b8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:03.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.940+0000 7fe0cf1c4640 1 --2- 172.21.15.57:0/3028051924 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fe0b8004810 0x7fe0b8004c90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4001750 tx=0x7fe0b401c3a0 comp rx=0 tx=0).stop 2024-04-25T23:54:03.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.940+0000 7fe0cf1c4640 1 -- 172.21.15.57:0/3028051924 shutdown_connections 2024-04-25T23:54:03.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.940+0000 7fe0cf1c4640 1 --2- 172.21.15.57:0/3028051924 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fe0b8004810 0x7fe0b8004c90 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:03.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.940+0000 7fe0cf1c4640 1 -- 172.21.15.57:0/3028051924 >> 172.21.15.57:0/3028051924 conn(0x7fe0c80b5a70 msgr2=0x7fe0c80b5e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:54:03.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.940+0000 7fe0cf1c4640 1 -- 172.21.15.57:0/3028051924 shutdown_connections 2024-04-25T23:54:03.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.940+0000 7fe0cf1c4640 1 -- 172.21.15.57:0/3028051924 wait complete. 2024-04-25T23:54:03.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.940+0000 7fe0cf1c4640 1 Processor -- start 2024-04-25T23:54:03.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.940+0000 7fe0cf1c4640 1 -- start start 2024-04-25T23:54:03.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.940+0000 7fe0cf1c4640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fe0b8004810 0x7fe0c8071320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:03.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.940+0000 7fe0cf1c4640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe0c8070440 0x7fe0c8137b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:03.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.940+0000 7fe0cf1c4640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0c80708a0 0x7fe0c8138040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:03.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.940+0000 7fe0cf1c4640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c80748f0 con 0x7fe0c8070440 2024-04-25T23:54:03.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.940+0000 7fe0cf1c4640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c8144fa0 con 0x7fe0c80708a0 2024-04-25T23:54:03.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.940+0000 7fe0cf1c4640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7fe0c8074a70 con 0x7fe0b8004810 2024-04-25T23:54:03.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.940+0000 7fe0ccf60640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fe0b8004810 0x7fe0c8071320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:03.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.940+0000 7fe0ccf60640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fe0b8004810 0x7fe0c8071320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3301/0 says I am v2:172.21.15.57:48810/0 (socket says 172.21.15.57:48810) 2024-04-25T23:54:03.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.940+0000 7fe0ccf60640 1 -- 172.21.15.57:0/742281428 learned_addr learned my addr 172.21.15.57:0/742281428 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:54:03.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.940+0000 7fe0c7fff640 1 --2- 172.21.15.57:0/742281428 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe0c8070440 0x7fe0c8137b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:03.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.940+0000 7fe0cd761640 1 --2- 172.21.15.57:0/742281428 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0c80708a0 0x7fe0c8138040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:03.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.940+0000 7fe0ccf60640 1 -- 172.21.15.57:0/742281428 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0c80708a0 msgr2=0x7fe0c8138040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:03.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.940+0000 7fe0ccf60640 1 --2- 172.21.15.57:0/742281428 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0c80708a0 0x7fe0c8138040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:03.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.940+0000 7fe0ccf60640 1 -- 172.21.15.57:0/742281428 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe0c8070440 msgr2=0x7fe0c8137b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:03.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.940+0000 7fe0ccf60640 1 --2- 172.21.15.57:0/742281428 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe0c8070440 0x7fe0c8137b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:03.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.940+0000 7fe0ccf60640 1 -- 172.21.15.57:0/742281428 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0c8138580 con 0x7fe0b8004810 2024-04-25T23:54:03.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.940+0000 7fe0cd761640 1 --2- 172.21.15.57:0/742281428 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0c80708a0 0x7fe0c8138040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T23:54:03.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.940+0000 7fe0ccf60640 1 --2- 172.21.15.57:0/742281428 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fe0b8004810 0x7fe0c8071320 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc007310 tx=0x7fe0bc0077d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:54:03.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.940+0000 7fe0c57fa640 1 -- 172.21.15.57:0/742281428 <== mon.2 v2:172.21.15.57:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc01a070 con 0x7fe0b8004810 2024-04-25T23:54:03.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.940+0000 7fe0c57fa640 1 -- 172.21.15.57:0/742281428 <== mon.2 v2:172.21.15.57:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0bc00f9c0 con 0x7fe0b8004810 2024-04-25T23:54:03.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.940+0000 7fe0cf1c4640 1 -- 172.21.15.57:0/742281428 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0c8070d70 con 0x7fe0b8004810 2024-04-25T23:54:03.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.944+0000 7fe0c57fa640 1 -- 172.21.15.57:0/742281428 <== mon.2 v2:172.21.15.57:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc00fd90 con 0x7fe0b8004810 2024-04-25T23:54:03.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.944+0000 7fe0cf1c4640 1 -- 172.21.15.57:0/742281428 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0c81debd0 con 0x7fe0b8004810 2024-04-25T23:54:03.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.944+0000 7fe0b2ffd640 1 -- 172.21.15.57:0/742281428 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0a0005090 con 0x7fe0b8004810 2024-04-25T23:54:03.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.944+0000 7fe0c57fa640 1 -- 172.21.15.57:0/742281428 <== mon.2 v2:172.21.15.57:3301/0 4 ==== mgrmap(e 5) ==== 87410+0+0 (secure 0 0 0) 0x7fe0bc01d070 con 0x7fe0b8004810 2024-04-25T23:54:03.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.944+0000 7fe0c57fa640 1 --2- 172.21.15.57:0/742281428 >> v2:172.21.15.80:6816/2493457677 conn(0x7fe0ac0678b0 0x7fe0ac069d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:03.930 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.944+0000 7fe0c57fa640 1 -- 172.21.15.57:0/742281428 <== mon.2 v2:172.21.15.57:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 4525+0+0 (secure 0 0 0) 0x7fe0bc085080 con 0x7fe0b8004810 2024-04-25T23:54:03.930 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.944+0000 7fe0c7fff640 1 --2- 172.21.15.57:0/742281428 >> v2:172.21.15.80:6816/2493457677 conn(0x7fe0ac0678b0 0x7fe0ac069d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:03.930 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.944+0000 7fe0c7fff640 1 --2- 172.21.15.57:0/742281428 >> v2:172.21.15.80:6816/2493457677 conn(0x7fe0ac0678b0 0x7fe0ac069d70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe0b80057f0 tx=0x7fe0b40011a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:54:03.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:03.948+0000 7fe0c57fa640 1 -- 172.21.15.57:0/742281428 <== mon.2 v2:172.21.15.57:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe0c8072420 con 0x7fe0b8004810 2024-04-25T23:54:04.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.128+0000 7fe0b2ffd640 1 -- 172.21.15.57:0/742281428 --> v2:172.21.15.80:6816/2493457677 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7fe0a0000bd0 con 0x7fe0ac0678b0 2024-04-25T23:54:04.113 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.128+0000 7fe0c57fa640 1 -- 172.21.15.57:0/742281428 <== mgr.4100 v2:172.21.15.80:6816/2493457677 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fe0a0000bd0 con 0x7fe0ac0678b0 2024-04-25T23:54:04.113 INFO:teuthology.orchestra.run.smithi057.stderr:instructed osd(s) 4 to deep-scrub 2024-04-25T23:54:04.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.132+0000 7fe0b2ffd640 1 -- 172.21.15.57:0/742281428 >> v2:172.21.15.80:6816/2493457677 conn(0x7fe0ac0678b0 msgr2=0x7fe0ac069d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:04.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.132+0000 7fe0b2ffd640 1 --2- 172.21.15.57:0/742281428 >> v2:172.21.15.80:6816/2493457677 conn(0x7fe0ac0678b0 0x7fe0ac069d70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe0b80057f0 tx=0x7fe0b40011a0 comp rx=0 tx=0).stop 2024-04-25T23:54:04.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.132+0000 7fe0b2ffd640 1 -- 172.21.15.57:0/742281428 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fe0b8004810 msgr2=0x7fe0c8071320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:04.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.132+0000 7fe0b2ffd640 1 --2- 172.21.15.57:0/742281428 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fe0b8004810 0x7fe0c8071320 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc007310 tx=0x7fe0bc0077d0 comp rx=0 tx=0).stop 2024-04-25T23:54:04.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.132+0000 7fe0b2ffd640 1 -- 172.21.15.57:0/742281428 shutdown_connections 2024-04-25T23:54:04.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.132+0000 7fe0b2ffd640 1 --2- 172.21.15.57:0/742281428 >> v2:172.21.15.80:6816/2493457677 conn(0x7fe0ac0678b0 0x7fe0ac069d70 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:04.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.132+0000 7fe0b2ffd640 1 --2- 172.21.15.57:0/742281428 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0c80708a0 0x7fe0c8138040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:04.117 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.132+0000 7fe0b2ffd640 1 --2- 172.21.15.57:0/742281428 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe0c8070440 0x7fe0c8137b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:04.117 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.132+0000 7fe0b2ffd640 1 --2- 172.21.15.57:0/742281428 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fe0b8004810 0x7fe0c8071320 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:04.117 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.132+0000 7fe0b2ffd640 1 -- 172.21.15.57:0/742281428 >> 172.21.15.57:0/742281428 conn(0x7fe0c80b5a70 msgr2=0x7fe0c813b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:54:04.117 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.132+0000 7fe0b2ffd640 1 -- 172.21.15.57:0/742281428 shutdown_connections 2024-04-25T23:54:04.117 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.132+0000 7fe0b2ffd640 1 -- 172.21.15.57:0/742281428 wait complete. 2024-04-25T23:54:04.134 INFO:tasks.ceph:Scrubbing osd.5 2024-04-25T23:54:04.134 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-25T23:54:04.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.280+0000 7f7c5d0f6640 1 Processor -- start 2024-04-25T23:54:04.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.280+0000 7f7c5d0f6640 1 -- start start 2024-04-25T23:54:04.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.284+0000 7f7c5d0f6640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f7c580a9b30 0x7f7c58143fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:04.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.284+0000 7f7c5d0f6640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f7c580b2090 0x7f7c58144510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:04.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.284+0000 7f7c5d0f6640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c58074f30 con 0x7f7c58074b50 2024-04-25T23:54:04.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.284+0000 7f7c5d0f6640 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 -- 0x7f7c580aa7c0 con 0x7f7c580b2090 2024-04-25T23:54:04.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.284+0000 7f7c5d0f6640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f7c58141af0 con 0x7f7c580a9b30 2024-04-25T23:54:04.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.284+0000 7f7c5759e640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f7c580b2090 0x7f7c58144510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:04.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.284+0000 7f7c5759e640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f7c580b2090 0x7f7c58144510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37276/0 (socket says 172.21.15.57:37276) 2024-04-25T23:54:04.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.284+0000 7f7c5759e640 1 -- 172.21.15.57:0/3129707253 learned_addr learned my addr 172.21.15.57:0/3129707253 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:54:04.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.284+0000 7f7c5659c640 1 --2- 172.21.15.57:0/3129707253 >> v2:172.21.15.80:3300/0 conn(0x7f7c580a9b30 0x7f7c58143fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:04.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.284+0000 7f7c5759e640 1 -- 172.21.15.57:0/3129707253 >> v2:172.21.15.80:3300/0 conn(0x7f7c580a9b30 msgr2=0x7f7c58143fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:54:04.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.284+0000 7f7c5759e640 1 --2- 172.21.15.57:0/3129707253 >> v2:172.21.15.80:3300/0 conn(0x7f7c580a9b30 0x7f7c58143fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:04.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.284+0000 7f7c5759e640 1 -- 172.21.15.57:0/3129707253 >> v1:172.21.15.80:6789/0 conn(0x7f7c58074b50 legacy=0x7f7c5813a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:04.268 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.284+0000 7f7c5759e640 1 -- 172.21.15.57:0/3129707253 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c5813ab70 con 0x7f7c580b2090 2024-04-25T23:54:04.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.284+0000 7f7c5759e640 1 --2- 172.21.15.57:0/3129707253 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c580b2090 0x7f7c58144510 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7c580ab130 tx=0x7f7c5001c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e27bc156cbb8e0f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:54:04.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.284+0000 7f7c55d9b640 1 -- 172.21.15.57:0/3129707253 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c500a5710 con 0x7f7c580b2090 2024-04-25T23:54:04.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.284+0000 7f7c55d9b640 1 -- 172.21.15.57:0/3129707253 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c500a58b0 con 0x7f7c580b2090 2024-04-25T23:54:04.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.284+0000 7f7c55d9b640 1 -- 172.21.15.57:0/3129707253 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c500a5c80 con 0x7f7c580b2090 2024-04-25T23:54:04.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.284+0000 7f7c5d0f6640 1 -- 172.21.15.57:0/3129707253 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c580b2090 msgr2=0x7f7c58144510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:04.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.284+0000 7f7c5d0f6640 1 --2- 172.21.15.57:0/3129707253 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c580b2090 0x7f7c58144510 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7c580ab130 tx=0x7f7c5001c940 comp rx=0 tx=0).stop 2024-04-25T23:54:04.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.284+0000 7f7c5d0f6640 1 -- 172.21.15.57:0/3129707253 shutdown_connections 2024-04-25T23:54:04.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.284+0000 7f7c5d0f6640 1 --2- 172.21.15.57:0/3129707253 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c580b2090 0x7f7c58144510 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:04.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.284+0000 7f7c5d0f6640 1 --2- 172.21.15.57:0/3129707253 >> v2:172.21.15.80:3300/0 conn(0x7f7c580a9b30 0x7f7c58143fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:04.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.284+0000 7f7c5d0f6640 1 -- 172.21.15.57:0/3129707253 >> 172.21.15.57:0/3129707253 conn(0x7f7c5806cf20 msgr2=0x7f7c580b59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:54:04.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.284+0000 7f7c5d0f6640 1 -- 172.21.15.57:0/3129707253 shutdown_connections 2024-04-25T23:54:04.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.284+0000 7f7c5d0f6640 1 -- 172.21.15.57:0/3129707253 wait complete. 2024-04-25T23:54:04.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.288+0000 7f7c5d0f6640 1 Processor -- start 2024-04-25T23:54:04.272 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.288+0000 7f7c5d0f6640 1 -- start start 2024-04-25T23:54:04.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.288+0000 7f7c5d0f6640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c58074b50 0x7f7c58143180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:04.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.288+0000 7f7c5d0f6640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c580a9b30 0x7f7c581dc240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:04.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.288+0000 7f7c5d0f6640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f7c580b2090 0x7f7c581dc780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:04.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.288+0000 7f7c5d0f6640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5814df10 con 0x7f7c580a9b30 2024-04-25T23:54:04.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.288+0000 7f7c5d0f6640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5814dd90 con 0x7f7c58074b50 2024-04-25T23:54:04.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.288+0000 7f7c5d0f6640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f7c5814e090 con 0x7f7c580b2090 2024-04-25T23:54:04.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.288+0000 7f7c5659c640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c580a9b30 0x7f7c581dc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:04.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.288+0000 7f7c5659c640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c580a9b30 0x7f7c581dc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37292/0 (socket says 172.21.15.57:37292) 2024-04-25T23:54:04.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.288+0000 7f7c5659c640 1 -- 172.21.15.57:0/2554458041 learned_addr learned my addr 172.21.15.57:0/2554458041 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:54:04.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.288+0000 7f7c5759e640 1 --2- 172.21.15.57:0/2554458041 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f7c580b2090 0x7f7c581dc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:04.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.288+0000 7f7c56d9d640 1 --2- 172.21.15.57:0/2554458041 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c58074b50 0x7f7c58143180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:04.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.288+0000 7f7c5659c640 1 -- 172.21.15.57:0/2554458041 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f7c580b2090 msgr2=0x7f7c581dc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:04.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.288+0000 7f7c5659c640 1 --2- 172.21.15.57:0/2554458041 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f7c580b2090 0x7f7c581dc780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:04.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.288+0000 7f7c5659c640 1 -- 172.21.15.57:0/2554458041 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c58074b50 msgr2=0x7f7c58143180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:04.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.288+0000 7f7c5659c640 1 --2- 172.21.15.57:0/2554458041 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c58074b50 0x7f7c58143180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:04.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.288+0000 7f7c5659c640 1 -- 172.21.15.57:0/2554458041 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c581dccc0 con 0x7f7c580a9b30 2024-04-25T23:54:04.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.288+0000 7f7c5659c640 1 --2- 172.21.15.57:0/2554458041 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c580a9b30 0x7f7c581dc240 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c00d800 tx=0x7f7c4c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:54:04.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.288+0000 7f7c3f7fe640 1 -- 172.21.15.57:0/2554458041 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c4c00b6c0 con 0x7f7c580a9b30 2024-04-25T23:54:04.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.288+0000 7f7c3f7fe640 1 -- 172.21.15.57:0/2554458041 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c4c019070 con 0x7f7c580a9b30 2024-04-25T23:54:04.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.288+0000 7f7c5d0f6640 1 -- 172.21.15.57:0/2554458041 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c5813e750 con 0x7f7c580a9b30 2024-04-25T23:54:04.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.288+0000 7f7c3f7fe640 1 -- 172.21.15.57:0/2554458041 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c4c00bae0 con 0x7f7c580a9b30 2024-04-25T23:54:04.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.288+0000 7f7c5d0f6640 1 -- 172.21.15.57:0/2554458041 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c5813d080 con 0x7f7c580a9b30 2024-04-25T23:54:04.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.288+0000 7f7c5d0f6640 1 -- 172.21.15.57:0/2554458041 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7c580761e0 con 0x7f7c580a9b30 2024-04-25T23:54:04.276 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.292+0000 7f7c3f7fe640 1 -- 172.21.15.57:0/2554458041 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) ==== 87410+0+0 (secure 0 0 0) 0x7f7c4c00bc80 con 0x7f7c580a9b30 2024-04-25T23:54:04.276 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.292+0000 7f7c3f7fe640 1 --2- 172.21.15.57:0/2554458041 >> v2:172.21.15.80:6816/2493457677 conn(0x7f7c44067880 0x7f7c44069d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:04.276 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.292+0000 7f7c3f7fe640 1 -- 172.21.15.57:0/2554458041 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 4525+0+0 (secure 0 0 0) 0x7f7c4c0865e0 con 0x7f7c580a9b30 2024-04-25T23:54:04.276 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.292+0000 7f7c3f7fe640 1 --2- 172.21.15.57:0/2554458041 >> v2:172.21.15.80:6811/2954555099 conn(0x7f7c4406e5e0 0x7f7c44070a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:04.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.292+0000 7f7c3f7fe640 1 -- 172.21.15.57:0/2554458041 --> v2:172.21.15.80:6811/2954555099 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7c4c024e50 con 0x7f7c4406e5e0 2024-04-25T23:54:04.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.292+0000 7f7c3f7fe640 1 -- 172.21.15.57:0/2554458041 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f7c4c086a70 con 0x7f7c580a9b30 2024-04-25T23:54:04.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.292+0000 7f7c56d9d640 1 --2- 172.21.15.57:0/2554458041 >> v2:172.21.15.80:6816/2493457677 conn(0x7f7c44067880 0x7f7c44069d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:04.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.292+0000 7f7c5759e640 1 --2- 172.21.15.57:0/2554458041 >> v2:172.21.15.80:6811/2954555099 conn(0x7f7c4406e5e0 0x7f7c44070a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:04.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.292+0000 7f7c56d9d640 1 --2- 172.21.15.57:0/2554458041 >> v2:172.21.15.80:6816/2493457677 conn(0x7f7c44067880 0x7f7c44069d40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7c48000f70 tx=0x7f7c48009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:54:04.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.292+0000 7f7c5759e640 1 --2- 172.21.15.57:0/2554458041 >> v2:172.21.15.80:6811/2954555099 conn(0x7f7c4406e5e0 0x7f7c44070a00 crc :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:54:04.278 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.292+0000 7f7c3f7fe640 1 -- 172.21.15.57:0/2554458041 <== osd.5 v2:172.21.15.80:6811/2954555099 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7c4c024e50 con 0x7f7c4406e5e0 2024-04-25T23:54:04.291 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.308+0000 7f7c5d0f6640 1 -- 172.21.15.57:0/2554458041 --> v2:172.21.15.80:6811/2954555099 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f7c58133560 con 0x7f7c4406e5e0 2024-04-25T23:54:04.292 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.308+0000 7f7c3f7fe640 1 -- 172.21.15.57:0/2554458041 <== osd.5 v2:172.21.15.80:6811/2954555099 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f7c58133560 con 0x7f7c4406e5e0 2024-04-25T23:54:04.293 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-25T23:54:04.293 INFO:teuthology.orchestra.run.smithi057.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T23:54:04.293 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-25T23:54:04.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.308+0000 7f7c5d0f6640 1 -- 172.21.15.57:0/2554458041 >> v2:172.21.15.80:6811/2954555099 conn(0x7f7c4406e5e0 msgr2=0x7f7c44070a00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:04.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.308+0000 7f7c5d0f6640 1 --2- 172.21.15.57:0/2554458041 >> v2:172.21.15.80:6811/2954555099 conn(0x7f7c4406e5e0 0x7f7c44070a00 crc :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:04.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.308+0000 7f7c5d0f6640 1 -- 172.21.15.57:0/2554458041 >> v2:172.21.15.80:6816/2493457677 conn(0x7f7c44067880 msgr2=0x7f7c44069d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:04.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.308+0000 7f7c5d0f6640 1 --2- 172.21.15.57:0/2554458041 >> v2:172.21.15.80:6816/2493457677 conn(0x7f7c44067880 0x7f7c44069d40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7c48000f70 tx=0x7f7c48009040 comp rx=0 tx=0).stop 2024-04-25T23:54:04.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.308+0000 7f7c5d0f6640 1 -- 172.21.15.57:0/2554458041 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c580a9b30 msgr2=0x7f7c581dc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:04.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.308+0000 7f7c5d0f6640 1 --2- 172.21.15.57:0/2554458041 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c580a9b30 0x7f7c581dc240 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c00d800 tx=0x7f7c4c00dcc0 comp rx=0 tx=0).stop 2024-04-25T23:54:04.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.308+0000 7f7c5759e640 1 -- 172.21.15.57:0/2554458041 reap_dead start 2024-04-25T23:54:04.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.308+0000 7f7c5d0f6640 1 -- 172.21.15.57:0/2554458041 shutdown_connections 2024-04-25T23:54:04.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.308+0000 7f7c5d0f6640 1 -- 172.21.15.57:0/2554458041 >> 172.21.15.57:0/2554458041 conn(0x7f7c5806cf20 msgr2=0x7f7c58071ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:54:04.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.308+0000 7f7c5d0f6640 1 -- 172.21.15.57:0/2554458041 shutdown_connections 2024-04-25T23:54:04.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.308+0000 7f7c5d0f6640 1 -- 172.21.15.57:0/2554458041 wait complete. 2024-04-25T23:54:04.311 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-25T23:54:04.429 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.444+0000 7f8f37042640 1 Processor -- start 2024-04-25T23:54:04.429 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.444+0000 7f8f37042640 1 -- start start 2024-04-25T23:54:04.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.444+0000 7f8f37042640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f8f30071d10 0x7f8f30070f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:04.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.444+0000 7f8f37042640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f30142c60 con 0x7f8f3013cda0 2024-04-25T23:54:04.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.444+0000 7f8f37042640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f300720f0 con 0x7f8f3006ea70 2024-04-25T23:54:04.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.444+0000 7f8f37042640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f8f30071450 con 0x7f8f30071d10 2024-04-25T23:54:04.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.444+0000 7f8f355df640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f8f3013cda0 0x7f8f30143110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:57828/0 (socket says 172.21.15.57:57828) 2024-04-25T23:54:04.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.444+0000 7f8f355df640 1 -- 172.21.15.57:0/2971217521 learned_addr learned my addr 172.21.15.57:0/2971217521 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-04-25T23:54:04.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.444+0000 7f8f34dde640 1 --2- 172.21.15.57:0/2971217521 >> v2:172.21.15.80:3300/0 conn(0x7f8f30071d10 0x7f8f30070f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:04.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.444+0000 7f8f2f7fe640 1 -- 172.21.15.57:0/2971217521 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3313181578 0 0) 0x7f8f30142c60 con 0x7f8f3013cda0 2024-04-25T23:54:04.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.444+0000 7f8f2f7fe640 1 -- 172.21.15.57:0/2971217521 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f20003530 con 0x7f8f3013cda0 2024-04-25T23:54:04.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.448+0000 7f8f2f7fe640 1 -- 172.21.15.57:0/2971217521 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 772574285 0 0) 0x7f8f20003530 con 0x7f8f3013cda0 2024-04-25T23:54:04.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.448+0000 7f8f2f7fe640 1 -- 172.21.15.57:0/2971217521 >> v2:172.21.15.80:3300/0 conn(0x7f8f30071d10 msgr2=0x7f8f30070f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:54:04.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.448+0000 7f8f2f7fe640 1 --2- 172.21.15.57:0/2971217521 >> v2:172.21.15.80:3300/0 conn(0x7f8f30071d10 0x7f8f30070f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:04.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.448+0000 7f8f2f7fe640 1 -- 172.21.15.57:0/2971217521 >> v1:172.21.15.80:6789/0 conn(0x7f8f3006ea70 legacy=0x7f8f3013c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:04.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.448+0000 7f8f2f7fe640 1 -- 172.21.15.57:0/2971217521 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f300adaa0 con 0x7f8f3013cda0 2024-04-25T23:54:04.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.448+0000 7f8f2f7fe640 1 -- 172.21.15.57:0/2971217521 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2627952221 0 0) 0x7f8f28001a00 con 0x7f8f3013cda0 2024-04-25T23:54:04.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.448+0000 7f8f2f7fe640 1 -- 172.21.15.57:0/2971217521 >> v1:172.21.15.57:6789/0 conn(0x7f8f3013cda0 legacy=0x7f8f30143110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:04.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.448+0000 7f8f2f7fe640 1 --2- 172.21.15.57:0/2971217521 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8f200039c0 0x7f8f20003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:04.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.448+0000 7f8f2f7fe640 1 --2- 172.21.15.57:0/2971217521 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f20004810 0x7f8f20004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:04.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.448+0000 7f8f2f7fe640 1 --2- 172.21.15.57:0/2971217521 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f8f200051d0 0x7f8f2000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:04.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.448+0000 7f8f2f7fe640 1 -- 172.21.15.57:0/2971217521 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f8f30071450 con 0x7f8f20004810 2024-04-25T23:54:04.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.448+0000 7f8f2f7fe640 1 -- 172.21.15.57:0/2971217521 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2000c170 con 0x7f8f200039c0 2024-04-25T23:54:04.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.448+0000 7f8f2f7fe640 1 -- 172.21.15.57:0/2971217521 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f8f2000c370 con 0x7f8f200051d0 2024-04-25T23:54:04.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.448+0000 7f8f355df640 1 --2- 172.21.15.57:0/2971217521 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f8f200051d0 0x7f8f2000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:04.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.448+0000 7f8f2ffff640 1 --2- 172.21.15.57:0/2971217521 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f20004810 0x7f8f20004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:04.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.448+0000 7f8f34dde640 1 --2- 172.21.15.57:0/2971217521 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8f200039c0 0x7f8f20003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:04.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.448+0000 7f8f355df640 1 -- 172.21.15.57:0/2971217521 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8f200039c0 msgr2=0x7f8f20003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:54:04.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.448+0000 7f8f355df640 1 --2- 172.21.15.57:0/2971217521 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8f200039c0 0x7f8f20003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:04.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.448+0000 7f8f355df640 1 -- 172.21.15.57:0/2971217521 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f20004810 msgr2=0x7f8f20004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:54:04.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.448+0000 7f8f355df640 1 --2- 172.21.15.57:0/2971217521 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f20004810 0x7f8f20004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:04.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.448+0000 7f8f355df640 1 -- 172.21.15.57:0/2971217521 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f20003530 con 0x7f8f200051d0 2024-04-25T23:54:04.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.448+0000 7f8f2ffff640 1 --2- 172.21.15.57:0/2971217521 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f20004810 0x7f8f20004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T23:54:04.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.448+0000 7f8f355df640 1 -- 172.21.15.57:0/2971217521 reap_dead start 2024-04-25T23:54:04.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.448+0000 7f8f355df640 1 --2- 172.21.15.57:0/2971217521 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f8f200051d0 0x7f8f2000b9d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8f2800a860 tx=0x7f8f2801d360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6e251ed8d5aef86 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:54:04.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.448+0000 7f8f2f7fe640 1 -- 172.21.15.57:0/2971217521 <== mon.2 v2:172.21.15.57:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c0013a0 con 0x7f8f200051d0 2024-04-25T23:54:04.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.448+0000 7f8f2f7fe640 1 -- 172.21.15.57:0/2971217521 <== mon.2 v2:172.21.15.57:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f280ad630 con 0x7f8f200051d0 2024-04-25T23:54:04.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.448+0000 7f8f2f7fe640 1 -- 172.21.15.57:0/2971217521 <== mon.2 v2:172.21.15.57:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2800a480 con 0x7f8f200051d0 2024-04-25T23:54:04.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.448+0000 7f8f37042640 1 -- 172.21.15.57:0/2971217521 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f8f200051d0 msgr2=0x7f8f2000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:04.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.448+0000 7f8f37042640 1 --2- 172.21.15.57:0/2971217521 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f8f200051d0 0x7f8f2000b9d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8f2800a860 tx=0x7f8f2801d360 comp rx=0 tx=0).stop 2024-04-25T23:54:04.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.448+0000 7f8f37042640 1 -- 172.21.15.57:0/2971217521 shutdown_connections 2024-04-25T23:54:04.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.448+0000 7f8f37042640 1 --2- 172.21.15.57:0/2971217521 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f8f200051d0 0x7f8f2000b9d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:04.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.448+0000 7f8f37042640 1 -- 172.21.15.57:0/2971217521 >> 172.21.15.57:0/2971217521 conn(0x7f8f300b90a0 msgr2=0x7f8f300b94b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:54:04.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.448+0000 7f8f37042640 1 -- 172.21.15.57:0/2971217521 shutdown_connections 2024-04-25T23:54:04.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.448+0000 7f8f37042640 1 -- 172.21.15.57:0/2971217521 wait complete. 2024-04-25T23:54:04.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.452+0000 7f8f37042640 1 Processor -- start 2024-04-25T23:54:04.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.452+0000 7f8f37042640 1 -- start start 2024-04-25T23:54:04.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.452+0000 7f8f37042640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f8f200051d0 0x7f8f3013e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:04.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.452+0000 7f8f37042640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8f3013d180 0x7f8f3013db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:04.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.452+0000 7f8f37042640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f301363c0 0x7f8f30136820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:04.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.452+0000 7f8f37042640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f8f300712a0 con 0x7f8f301363c0 2024-04-25T23:54:04.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.452+0000 7f8f37042640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8f30071120 con 0x7f8f3013d180 2024-04-25T23:54:04.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.452+0000 7f8f37042640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f8f3013c690 con 0x7f8f200051d0 2024-04-25T23:54:04.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.452+0000 7f8f34dde640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f8f200051d0 0x7f8f3013e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:04.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.452+0000 7f8f355df640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f301363c0 0x7f8f30136820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:04.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.452+0000 7f8f355df640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f301363c0 0x7f8f30136820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38068/0 (socket says 172.21.15.57:38068) 2024-04-25T23:54:04.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.452+0000 7f8f355df640 1 -- 172.21.15.57:0/2050981239 learned_addr learned my addr 172.21.15.57:0/2050981239 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:54:04.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.452+0000 7f8f34dde640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f8f200051d0 0x7f8f3013e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3301/0 says I am v2:172.21.15.57:59858/0 (socket says 172.21.15.57:59858) 2024-04-25T23:54:04.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.452+0000 7f8f2ffff640 1 --2- 172.21.15.57:0/2050981239 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8f3013d180 0x7f8f3013db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:04.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.452+0000 7f8f355df640 1 -- 172.21.15.57:0/2050981239 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f8f200051d0 msgr2=0x7f8f3013e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:04.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.452+0000 7f8f355df640 1 --2- 172.21.15.57:0/2050981239 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f8f200051d0 0x7f8f3013e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:04.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.452+0000 7f8f355df640 1 -- 172.21.15.57:0/2050981239 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8f3013d180 msgr2=0x7f8f3013db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:04.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.452+0000 7f8f355df640 1 --2- 172.21.15.57:0/2050981239 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8f3013d180 0x7f8f3013db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:04.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.452+0000 7f8f355df640 1 -- 172.21.15.57:0/2050981239 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f30136d60 con 0x7f8f301363c0 2024-04-25T23:54:04.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.452+0000 7f8f355df640 1 --2- 172.21.15.57:0/2050981239 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f301363c0 0x7f8f30136820 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8f2801dc00 tx=0x7f8f20005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:54:04.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.452+0000 7f8f2d7fa640 1 -- 172.21.15.57:0/2050981239 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f280bc070 con 0x7f8f301363c0 2024-04-25T23:54:04.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.452+0000 7f8f2d7fa640 1 -- 172.21.15.57:0/2050981239 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f28001800 con 0x7f8f301363c0 2024-04-25T23:54:04.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.452+0000 7f8f37042640 1 -- 172.21.15.57:0/2050981239 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f3013f5b0 con 0x7f8f301363c0 2024-04-25T23:54:04.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.452+0000 7f8f2d7fa640 1 -- 172.21.15.57:0/2050981239 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2800a480 con 0x7f8f301363c0 2024-04-25T23:54:04.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.452+0000 7f8f37042640 1 -- 172.21.15.57:0/2050981239 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f30131250 con 0x7f8f301363c0 2024-04-25T23:54:04.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.452+0000 7f8f37042640 1 -- 172.21.15.57:0/2050981239 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f300761e0 con 0x7f8f301363c0 2024-04-25T23:54:04.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.456+0000 7f8f2d7fa640 1 -- 172.21.15.57:0/2050981239 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) ==== 87410+0+0 (secure 0 0 0) 0x7f8f280bf070 con 0x7f8f301363c0 2024-04-25T23:54:04.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.456+0000 7f8f2d7fa640 1 --2- 172.21.15.57:0/2050981239 >> v2:172.21.15.80:6816/2493457677 conn(0x7f8f140678b0 0x7f8f14069d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:04.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.456+0000 7f8f2d7fa640 1 -- 172.21.15.57:0/2050981239 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 4525+0+0 (secure 0 0 0) 0x7f8f2811fd40 con 0x7f8f301363c0 2024-04-25T23:54:04.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.456+0000 7f8f34dde640 1 --2- 172.21.15.57:0/2050981239 >> v2:172.21.15.80:6816/2493457677 conn(0x7f8f140678b0 0x7f8f14069d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:04.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.456+0000 7f8f34dde640 1 --2- 172.21.15.57:0/2050981239 >> v2:172.21.15.80:6816/2493457677 conn(0x7f8f140678b0 0x7f8f14069d70 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8f240099c0 tx=0x7f8f24008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:54:04.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.460+0000 7f8f2d7fa640 1 -- 172.21.15.57:0/2050981239 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8f280c6070 con 0x7f8f301363c0 2024-04-25T23:54:04.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.664+0000 7f8f37042640 1 -- 172.21.15.57:0/2050981239 --> v2:172.21.15.80:6816/2493457677 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f8f3013e150 con 0x7f8f140678b0 2024-04-25T23:54:04.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.668+0000 7f8f2d7fa640 1 -- 172.21.15.57:0/2050981239 <== mgr.4100 v2:172.21.15.80:6816/2493457677 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f8f3013e150 con 0x7f8f140678b0 2024-04-25T23:54:04.651 INFO:teuthology.orchestra.run.smithi057.stderr:instructed osd(s) 5 to deep-scrub 2024-04-25T23:54:04.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.668+0000 7f8f37042640 1 -- 172.21.15.57:0/2050981239 >> v2:172.21.15.80:6816/2493457677 conn(0x7f8f140678b0 msgr2=0x7f8f14069d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:04.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.668+0000 7f8f37042640 1 --2- 172.21.15.57:0/2050981239 >> v2:172.21.15.80:6816/2493457677 conn(0x7f8f140678b0 0x7f8f14069d70 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8f240099c0 tx=0x7f8f24008040 comp rx=0 tx=0).stop 2024-04-25T23:54:04.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.668+0000 7f8f37042640 1 -- 172.21.15.57:0/2050981239 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f301363c0 msgr2=0x7f8f30136820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:04.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.668+0000 7f8f37042640 1 --2- 172.21.15.57:0/2050981239 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f301363c0 0x7f8f30136820 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8f2801dc00 tx=0x7f8f20005840 comp rx=0 tx=0).stop 2024-04-25T23:54:04.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.668+0000 7f8f37042640 1 -- 172.21.15.57:0/2050981239 shutdown_connections 2024-04-25T23:54:04.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.668+0000 7f8f37042640 1 --2- 172.21.15.57:0/2050981239 >> v2:172.21.15.80:6816/2493457677 conn(0x7f8f140678b0 0x7f8f14069d70 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:04.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.668+0000 7f8f37042640 1 --2- 172.21.15.57:0/2050981239 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8f3013d180 0x7f8f3013db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:04.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.668+0000 7f8f37042640 1 --2- 172.21.15.57:0/2050981239 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f301363c0 0x7f8f30136820 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:04.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.668+0000 7f8f37042640 1 --2- 172.21.15.57:0/2050981239 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f8f200051d0 0x7f8f3013e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:04.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.668+0000 7f8f37042640 1 -- 172.21.15.57:0/2050981239 >> 172.21.15.57:0/2050981239 conn(0x7f8f300b90a0 msgr2=0x7f8f30143110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:54:04.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.668+0000 7f8f37042640 1 -- 172.21.15.57:0/2050981239 shutdown_connections 2024-04-25T23:54:04.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.668+0000 7f8f37042640 1 -- 172.21.15.57:0/2050981239 wait complete. 2024-04-25T23:54:04.677 INFO:tasks.ceph:Scrubbing osd.6 2024-04-25T23:54:04.677 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-25T23:54:04.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.852+0000 7f4653f63640 1 Processor -- start 2024-04-25T23:54:04.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.852+0000 7f4653f63640 1 -- start start 2024-04-25T23:54:04.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.852+0000 7f4653f63640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f464c13e260 0x7f464c1440a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:04.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.852+0000 7f4653f63640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f464c0b7830 con 0x7f464c0b7450 2024-04-25T23:54:04.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.852+0000 7f4653f63640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f464c0ab8e0 con 0x7f464c0a9b30 2024-04-25T23:54:04.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.852+0000 7f4653f63640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f464c0aa160 con 0x7f464c13e260 2024-04-25T23:54:04.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.852+0000 7f4651cff640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f464c0b7450 0x7f464c13db50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:57844/0 (socket says 172.21.15.57:57844) 2024-04-25T23:54:04.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.852+0000 7f4651cff640 1 -- 172.21.15.57:0/2926506178 learned_addr learned my addr 172.21.15.57:0/2926506178 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-04-25T23:54:04.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.852+0000 7f46514fe640 1 --2- 172.21.15.57:0/2926506178 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f464c13e260 0x7f464c1440a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:04.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.856+0000 7f4650cfd640 1 -- 172.21.15.57:0/2926506178 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1205573018 0 0) 0x7f4648001120 con 0x7f464c0b7450 2024-04-25T23:54:04.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.856+0000 7f46514fe640 1 -- 172.21.15.57:0/2926506178 >> v1:172.21.15.80:6789/0 conn(0x7f464c0a9b30 legacy=0x7f464c1445e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:54:04.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.856+0000 7f46514fe640 1 -- 172.21.15.57:0/2926506178 >> v1:172.21.15.57:6789/0 conn(0x7f464c0b7450 legacy=0x7f464c13db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:04.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.856+0000 7f46514fe640 1 -- 172.21.15.57:0/2926506178 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f464c0b3240 con 0x7f464c13e260 2024-04-25T23:54:04.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.856+0000 7f46514fe640 1 --2- 172.21.15.57:0/2926506178 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f464c13e260 0x7f464c1440a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f463c0097f0 tx=0x7f463c01c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=154f6d4dcde4da0f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:54:04.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.856+0000 7f4650cfd640 1 -- 172.21.15.57:0/2926506178 <== mon.2 v2:172.21.15.57:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f463c0b2070 con 0x7f464c13e260 2024-04-25T23:54:04.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.856+0000 7f4650cfd640 1 -- 172.21.15.57:0/2926506178 <== mon.2 v2:172.21.15.57:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f463c0ad640 con 0x7f464c13e260 2024-04-25T23:54:04.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.856+0000 7f4650cfd640 1 -- 172.21.15.57:0/2926506178 <== mon.2 v2:172.21.15.57:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f463c0ada10 con 0x7f464c13e260 2024-04-25T23:54:04.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.856+0000 7f4653f63640 1 -- 172.21.15.57:0/2926506178 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f464c13e260 msgr2=0x7f464c1440a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:04.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.856+0000 7f4653f63640 1 --2- 172.21.15.57:0/2926506178 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f464c13e260 0x7f464c1440a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f463c0097f0 tx=0x7f463c01c480 comp rx=0 tx=0).stop 2024-04-25T23:54:04.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.856+0000 7f4653f63640 1 -- 172.21.15.57:0/2926506178 shutdown_connections 2024-04-25T23:54:04.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.856+0000 7f4653f63640 1 --2- 172.21.15.57:0/2926506178 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f464c13e260 0x7f464c1440a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:04.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.856+0000 7f4653f63640 1 -- 172.21.15.57:0/2926506178 >> 172.21.15.57:0/2926506178 conn(0x7f464c0750a0 msgr2=0x7f464c0b6ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:54:04.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.856+0000 7f4653f63640 1 -- 172.21.15.57:0/2926506178 shutdown_connections 2024-04-25T23:54:04.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.856+0000 7f4653f63640 1 -- 172.21.15.57:0/2926506178 wait complete. 2024-04-25T23:54:04.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.860+0000 7f4653f63640 1 Processor -- start 2024-04-25T23:54:04.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.860+0000 7f4653f63640 1 -- start start 2024-04-25T23:54:04.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.860+0000 7f4653f63640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f464c0a9b30 0x7f464c138550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:04.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.860+0000 7f4653f63640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f464c0b7450 0x7f464c138a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:04.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.860+0000 7f4653f63640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f464c13e260 0x7f464c1dc700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:04.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.860+0000 7f4653f63640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f464c14f140 con 0x7f464c13e260 2024-04-25T23:54:04.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.860+0000 7f4653f63640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f464c14efc0 con 0x7f464c0b7450 2024-04-25T23:54:04.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.860+0000 7f4653f63640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f464c14f2c0 con 0x7f464c0a9b30 2024-04-25T23:54:04.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.860+0000 7f4652500640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f464c13e260 0x7f464c1dc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:04.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.860+0000 7f4651cff640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f464c0a9b30 0x7f464c138550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:04.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.860+0000 7f4652500640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f464c13e260 0x7f464c1dc700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38082/0 (socket says 172.21.15.57:38082) 2024-04-25T23:54:04.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.860+0000 7f4652500640 1 -- 172.21.15.57:0/2575617653 learned_addr learned my addr 172.21.15.57:0/2575617653 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:54:04.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.860+0000 7f46514fe640 1 --2- 172.21.15.57:0/2575617653 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f464c0b7450 0x7f464c138a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:04.846 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.860+0000 7f4651cff640 1 -- 172.21.15.57:0/2575617653 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f464c0b7450 msgr2=0x7f464c138a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:04.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.860+0000 7f4651cff640 1 --2- 172.21.15.57:0/2575617653 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f464c0b7450 0x7f464c138a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:04.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.860+0000 7f4651cff640 1 -- 172.21.15.57:0/2575617653 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f464c13e260 msgr2=0x7f464c1dc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:04.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.860+0000 7f4651cff640 1 --2- 172.21.15.57:0/2575617653 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f464c13e260 0x7f464c1dc700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:04.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.860+0000 7f4651cff640 1 -- 172.21.15.57:0/2575617653 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f464c0ac1d0 con 0x7f464c0a9b30 2024-04-25T23:54:04.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.860+0000 7f4652500640 1 --2- 172.21.15.57:0/2575617653 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f464c13e260 0x7f464c1dc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T23:54:04.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.860+0000 7f4651cff640 1 --2- 172.21.15.57:0/2575617653 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f464c0a9b30 0x7f464c138550 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f464800c780 tx=0x7f464800cc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:54:04.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.860+0000 7f46427fc640 1 -- 172.21.15.57:0/2575617653 <== mon.2 v2:172.21.15.57:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4648018070 con 0x7f464c0a9b30 2024-04-25T23:54:04.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.860+0000 7f46427fc640 1 -- 172.21.15.57:0/2575617653 <== mon.2 v2:172.21.15.57:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46480070d0 con 0x7f464c0a9b30 2024-04-25T23:54:04.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.860+0000 7f4653f63640 1 -- 172.21.15.57:0/2575617653 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f464c138fd0 con 0x7f464c0a9b30 2024-04-25T23:54:04.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.860+0000 7f4653f63640 1 -- 172.21.15.57:0/2575617653 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f464c1dcc40 con 0x7f464c0a9b30 2024-04-25T23:54:04.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.860+0000 7f46427fc640 1 -- 172.21.15.57:0/2575617653 <== mon.2 v2:172.21.15.57:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46480074c0 con 0x7f464c0a9b30 2024-04-25T23:54:04.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.864+0000 7f46427fc640 1 -- 172.21.15.57:0/2575617653 <== mon.2 v2:172.21.15.57:3301/0 4 ==== mgrmap(e 5) ==== 87410+0+0 (secure 0 0 0) 0x7f464801b070 con 0x7f464c0a9b30 2024-04-25T23:54:04.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.864+0000 7f46427fc640 1 --2- 172.21.15.57:0/2575617653 >> v2:172.21.15.80:6816/2493457677 conn(0x7f4630067880 0x7f4630069d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:04.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.864+0000 7f46427fc640 1 -- 172.21.15.57:0/2575617653 <== mon.2 v2:172.21.15.57:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 4525+0+0 (secure 0 0 0) 0x7f4648084a70 con 0x7f464c0a9b30 2024-04-25T23:54:04.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.864+0000 7f46514fe640 1 --2- 172.21.15.57:0/2575617653 >> v2:172.21.15.80:6816/2493457677 conn(0x7f4630067880 0x7f4630069d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:04.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.864+0000 7f46514fe640 1 --2- 172.21.15.57:0/2575617653 >> v2:172.21.15.80:6816/2493457677 conn(0x7f4630067880 0x7f4630069d40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f463c0af000 tx=0x7f463c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:54:04.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.864+0000 7f4653f63640 1 --2- 172.21.15.57:0/2575617653 >> v2:172.21.15.80:6801/163896711 conn(0x7f4634001640 0x7f4634003b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:04.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.864+0000 7f4653f63640 1 -- 172.21.15.57:0/2575617653 --> v2:172.21.15.80:6801/163896711 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4634006ab0 con 0x7f4634001640 2024-04-25T23:54:04.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.868+0000 7f4652500640 1 --2- 172.21.15.57:0/2575617653 >> v2:172.21.15.80:6801/163896711 conn(0x7f4634001640 0x7f4634003b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:04.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.868+0000 7f4652500640 1 --2- 172.21.15.57:0/2575617653 >> v2:172.21.15.80:6801/163896711 conn(0x7f4634001640 0x7f4634003b00 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:54:04.853 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.868+0000 7f46427fc640 1 -- 172.21.15.57:0/2575617653 <== osd.6 v2:172.21.15.80:6801/163896711 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4634006ab0 con 0x7f4634001640 2024-04-25T23:54:04.871 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.888+0000 7f4653f63640 1 -- 172.21.15.57:0/2575617653 --> v2:172.21.15.80:6801/163896711 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f4634005c10 con 0x7f4634001640 2024-04-25T23:54:04.871 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.888+0000 7f46427fc640 1 -- 172.21.15.57:0/2575617653 <== osd.6 v2:172.21.15.80:6801/163896711 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f4634005c10 con 0x7f4634001640 2024-04-25T23:54:04.872 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-25T23:54:04.872 INFO:teuthology.orchestra.run.smithi057.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T23:54:04.872 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-25T23:54:04.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.888+0000 7f4653f63640 1 -- 172.21.15.57:0/2575617653 >> v2:172.21.15.80:6801/163896711 conn(0x7f4634001640 msgr2=0x7f4634003b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:04.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.888+0000 7f4653f63640 1 --2- 172.21.15.57:0/2575617653 >> v2:172.21.15.80:6801/163896711 conn(0x7f4634001640 0x7f4634003b00 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:04.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.888+0000 7f4653f63640 1 -- 172.21.15.57:0/2575617653 >> v2:172.21.15.80:6816/2493457677 conn(0x7f4630067880 msgr2=0x7f4630069d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:04.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.888+0000 7f4653f63640 1 --2- 172.21.15.57:0/2575617653 >> v2:172.21.15.80:6816/2493457677 conn(0x7f4630067880 0x7f4630069d40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f463c0af000 tx=0x7f463c008040 comp rx=0 tx=0).stop 2024-04-25T23:54:04.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.888+0000 7f4653f63640 1 -- 172.21.15.57:0/2575617653 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f464c0a9b30 msgr2=0x7f464c138550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:04.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.888+0000 7f4653f63640 1 --2- 172.21.15.57:0/2575617653 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f464c0a9b30 0x7f464c138550 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f464800c780 tx=0x7f464800cc40 comp rx=0 tx=0).stop 2024-04-25T23:54:04.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.888+0000 7f4652500640 1 -- 172.21.15.57:0/2575617653 reap_dead start 2024-04-25T23:54:04.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.888+0000 7f4653f63640 1 -- 172.21.15.57:0/2575617653 shutdown_connections 2024-04-25T23:54:04.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.888+0000 7f4653f63640 1 -- 172.21.15.57:0/2575617653 >> 172.21.15.57:0/2575617653 conn(0x7f464c0750a0 msgr2=0x7f464c0b49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:54:04.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.888+0000 7f4653f63640 1 -- 172.21.15.57:0/2575617653 shutdown_connections 2024-04-25T23:54:04.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:04.888+0000 7f4653f63640 1 -- 172.21.15.57:0/2575617653 wait complete. 2024-04-25T23:54:04.893 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-25T23:54:05.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.032+0000 7f31a5416640 1 Processor -- start 2024-04-25T23:54:05.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.032+0000 7f31a5416640 1 -- start start 2024-04-25T23:54:05.020 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.032+0000 7f31a5416640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f31a01465d0 0x7f31a01469e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:05.020 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.032+0000 7f31a5416640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31a0146120 con 0x7f31a0074b50 2024-04-25T23:54:05.020 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.032+0000 7f31a5416640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31a0074f30 con 0x7f31a0142b20 2024-04-25T23:54:05.020 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.032+0000 7f31a5416640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f31a0146f20 con 0x7f31a01465d0 2024-04-25T23:54:05.020 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.032+0000 7f319effd640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f31a0074b50 0x7f31a0071810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:57846/0 (socket says 172.21.15.57:57846) 2024-04-25T23:54:05.020 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.032+0000 7f319effd640 1 -- 172.21.15.57:0/512228429 learned_addr learned my addr 172.21.15.57:0/512228429 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-04-25T23:54:05.020 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.032+0000 7f319f7fe640 1 --2- 172.21.15.57:0/512228429 >> v2:172.21.15.80:3300/0 conn(0x7f31a01465d0 0x7f31a01469e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:05.020 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.036+0000 7f319e7fc640 1 -- 172.21.15.57:0/512228429 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1485650302 0 0) 0x7f31a0146120 con 0x7f31a0074b50 2024-04-25T23:54:05.020 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.036+0000 7f319e7fc640 1 -- 172.21.15.57:0/512228429 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f318c003530 con 0x7f31a0074b50 2024-04-25T23:54:05.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.036+0000 7f319e7fc640 1 -- 172.21.15.57:0/512228429 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1458419386 0 0) 0x7f318c003530 con 0x7f31a0074b50 2024-04-25T23:54:05.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.036+0000 7f319e7fc640 1 -- 172.21.15.57:0/512228429 >> v2:172.21.15.80:3300/0 conn(0x7f31a01465d0 msgr2=0x7f31a01469e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:54:05.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.036+0000 7f319e7fc640 1 --2- 172.21.15.57:0/512228429 >> v2:172.21.15.80:3300/0 conn(0x7f31a01465d0 0x7f31a01469e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:05.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.036+0000 7f319e7fc640 1 -- 172.21.15.57:0/512228429 >> v1:172.21.15.80:6789/0 conn(0x7f31a0142b20 legacy=0x7f31a0142160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:05.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.036+0000 7f319e7fc640 1 -- 172.21.15.57:0/512228429 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31a0072c60 con 0x7f31a0074b50 2024-04-25T23:54:05.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.036+0000 7f319e7fc640 1 -- 172.21.15.57:0/512228429 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2627952221 0 0) 0x7f3190001680 con 0x7f31a0074b50 2024-04-25T23:54:05.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.036+0000 7f319e7fc640 1 -- 172.21.15.57:0/512228429 >> v1:172.21.15.57:6789/0 conn(0x7f31a0074b50 legacy=0x7f31a0071810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:05.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.036+0000 7f319e7fc640 1 --2- 172.21.15.57:0/512228429 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f318c003a10 0x7f318c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:05.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.036+0000 7f319e7fc640 1 --2- 172.21.15.57:0/512228429 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f318c004810 0x7f318c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:05.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.036+0000 7f319e7fc640 1 --2- 172.21.15.57:0/512228429 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f318c0051d0 0x7f318c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:05.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.036+0000 7f319e7fc640 1 -- 172.21.15.57:0/512228429 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f31a0146f20 con 0x7f318c003a10 2024-04-25T23:54:05.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.036+0000 7f319e7fc640 1 -- 172.21.15.57:0/512228429 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f318c00c0d0 con 0x7f318c004810 2024-04-25T23:54:05.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.036+0000 7f319e7fc640 1 -- 172.21.15.57:0/512228429 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f318c00c2d0 con 0x7f318c0051d0 2024-04-25T23:54:05.021 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.036+0000 7f319effd640 1 --2- 172.21.15.57:0/512228429 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f318c003a10 0x7f318c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:05.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.036+0000 7f319f7fe640 1 --2- 172.21.15.57:0/512228429 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f318c0051d0 0x7f318c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:05.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.036+0000 7f3197fff640 1 --2- 172.21.15.57:0/512228429 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f318c004810 0x7f318c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:05.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.036+0000 7f319effd640 1 -- 172.21.15.57:0/512228429 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f318c0051d0 msgr2=0x7f318c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:54:05.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.036+0000 7f319effd640 1 --2- 172.21.15.57:0/512228429 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f318c0051d0 0x7f318c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:05.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.036+0000 7f319effd640 1 -- 172.21.15.57:0/512228429 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f318c004810 msgr2=0x7f318c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:54:05.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.036+0000 7f319effd640 1 --2- 172.21.15.57:0/512228429 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f318c004810 0x7f318c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:05.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.036+0000 7f319effd640 1 -- 172.21.15.57:0/512228429 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f318c003530 con 0x7f318c003a10 2024-04-25T23:54:05.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.036+0000 7f319f7fe640 1 --2- 172.21.15.57:0/512228429 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f318c0051d0 0x7f318c00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T23:54:05.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.036+0000 7f3197fff640 1 --2- 172.21.15.57:0/512228429 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f318c004810 0x7f318c004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T23:54:05.022 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.036+0000 7f319f7fe640 1 -- 172.21.15.57:0/512228429 reap_dead start 2024-04-25T23:54:05.023 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.036+0000 7f319effd640 1 --2- 172.21.15.57:0/512228429 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f318c003a10 0x7f318c003e30 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f319000c110 tx=0x7f319001cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2998fa834e6a20d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:54:05.023 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.036+0000 7f319e7fc640 1 -- 172.21.15.57:0/512228429 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3190007c30 con 0x7f318c003a10 2024-04-25T23:54:05.023 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.036+0000 7f319e7fc640 1 -- 172.21.15.57:0/512228429 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31900a9020 con 0x7f318c003a10 2024-04-25T23:54:05.024 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.036+0000 7f319e7fc640 1 -- 172.21.15.57:0/512228429 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3190005ae0 con 0x7f318c003a10 2024-04-25T23:54:05.024 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.040+0000 7f31a5416640 1 -- 172.21.15.57:0/512228429 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f318c003a10 msgr2=0x7f318c003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:05.024 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.040+0000 7f31a5416640 1 --2- 172.21.15.57:0/512228429 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f318c003a10 0x7f318c003e30 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f319000c110 tx=0x7f319001cb00 comp rx=0 tx=0).stop 2024-04-25T23:54:05.024 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.040+0000 7f31a5416640 1 -- 172.21.15.57:0/512228429 shutdown_connections 2024-04-25T23:54:05.024 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.040+0000 7f31a5416640 1 --2- 172.21.15.57:0/512228429 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f318c003a10 0x7f318c003e30 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:05.024 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.040+0000 7f31a5416640 1 -- 172.21.15.57:0/512228429 >> 172.21.15.57:0/512228429 conn(0x7f31a00b58a0 msgr2=0x7f31a00b5cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:54:05.024 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.040+0000 7f31a5416640 1 -- 172.21.15.57:0/512228429 shutdown_connections 2024-04-25T23:54:05.024 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.040+0000 7f31a5416640 1 -- 172.21.15.57:0/512228429 wait complete. 2024-04-25T23:54:05.026 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.040+0000 7f31a5416640 1 Processor -- start 2024-04-25T23:54:05.026 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.040+0000 7f31a5416640 1 -- start start 2024-04-25T23:54:05.026 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.040+0000 7f31a5416640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f318c003a10 0x7f31a01d18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:05.026 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.040+0000 7f31a5416640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f31a01d23f0 0x7f31a0140fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:05.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.040+0000 7f31a5416640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f31a01d29f0 0x7f31a01d0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:05.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.040+0000 7f31a5416640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f31a006ffb0 con 0x7f318c003a10 2024-04-25T23:54:05.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.040+0000 7f31a5416640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f31a006fe30 con 0x7f31a01d23f0 2024-04-25T23:54:05.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.040+0000 7f31a5416640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f31a0070130 con 0x7f31a01d29f0 2024-04-25T23:54:05.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.040+0000 7f319effd640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f318c003a10 0x7f31a01d18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:05.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.040+0000 7f319f7fe640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f31a01d29f0 0x7f31a01d0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:05.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.040+0000 7f319effd640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f318c003a10 0x7f31a01d18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38104/0 (socket says 172.21.15.57:38104) 2024-04-25T23:54:05.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.040+0000 7f319effd640 1 -- 172.21.15.57:0/1333136503 learned_addr learned my addr 172.21.15.57:0/1333136503 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:54:05.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.040+0000 7f319f7fe640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f31a01d29f0 0x7f31a01d0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3301/0 says I am v2:172.21.15.57:59898/0 (socket says 172.21.15.57:59898) 2024-04-25T23:54:05.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.040+0000 7f3197fff640 1 --2- 172.21.15.57:0/1333136503 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f31a01d23f0 0x7f31a0140fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:05.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.040+0000 7f319effd640 1 -- 172.21.15.57:0/1333136503 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f31a01d29f0 msgr2=0x7f31a01d0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:05.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.040+0000 7f319effd640 1 --2- 172.21.15.57:0/1333136503 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f31a01d29f0 0x7f31a01d0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:05.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.040+0000 7f319effd640 1 -- 172.21.15.57:0/1333136503 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f31a01d23f0 msgr2=0x7f31a0140fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:05.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.040+0000 7f319effd640 1 --2- 172.21.15.57:0/1333136503 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f31a01d23f0 0x7f31a0140fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:05.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.040+0000 7f319effd640 1 -- 172.21.15.57:0/1333136503 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31a00ae950 con 0x7f318c003a10 2024-04-25T23:54:05.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.040+0000 7f319f7fe640 1 --2- 172.21.15.57:0/1333136503 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f31a01d29f0 0x7f31a01d0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T23:54:05.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.040+0000 7f319effd640 1 --2- 172.21.15.57:0/1333136503 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f318c003a10 0x7f31a01d18b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f319001c930 tx=0x7f318c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:54:05.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.040+0000 7f31977fe640 1 -- 172.21.15.57:0/1333136503 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31900b4070 con 0x7f318c003a10 2024-04-25T23:54:05.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.040+0000 7f31977fe640 1 -- 172.21.15.57:0/1333136503 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3190000ee0 con 0x7f318c003a10 2024-04-25T23:54:05.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.040+0000 7f31a5416640 1 -- 172.21.15.57:0/1333136503 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31a01d0760 con 0x7f318c003a10 2024-04-25T23:54:05.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.040+0000 7f31977fe640 1 -- 172.21.15.57:0/1333136503 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3190001b70 con 0x7f318c003a10 2024-04-25T23:54:05.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.040+0000 7f31a5416640 1 -- 172.21.15.57:0/1333136503 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31a01deeb0 con 0x7f318c003a10 2024-04-25T23:54:05.029 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.044+0000 7f31a5416640 1 -- 172.21.15.57:0/1333136503 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31a00761e0 con 0x7f318c003a10 2024-04-25T23:54:05.029 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.044+0000 7f31977fe640 1 -- 172.21.15.57:0/1333136503 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) ==== 87410+0+0 (secure 0 0 0) 0x7f31900bf060 con 0x7f318c003a10 2024-04-25T23:54:05.029 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.044+0000 7f31977fe640 1 --2- 172.21.15.57:0/1333136503 >> v2:172.21.15.80:6816/2493457677 conn(0x7f3180067880 0x7f3180069d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:05.029 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.044+0000 7f31977fe640 1 -- 172.21.15.57:0/1333136503 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 4525+0+0 (secure 0 0 0) 0x7f319011e580 con 0x7f318c003a10 2024-04-25T23:54:05.030 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.044+0000 7f3197fff640 1 --2- 172.21.15.57:0/1333136503 >> v2:172.21.15.80:6816/2493457677 conn(0x7f3180067880 0x7f3180069d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:05.030 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.044+0000 7f3197fff640 1 --2- 172.21.15.57:0/1333136503 >> v2:172.21.15.80:6816/2493457677 conn(0x7f3180067880 0x7f3180069d40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f31a00b1980 tx=0x7f3188009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:54:05.033 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.048+0000 7f31977fe640 1 -- 172.21.15.57:0/1333136503 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3190016810 con 0x7f318c003a10 2024-04-25T23:54:05.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.216+0000 7f31a5416640 1 -- 172.21.15.57:0/1333136503 --> v2:172.21.15.80:6816/2493457677 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f31a00b53a0 con 0x7f3180067880 2024-04-25T23:54:05.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.216+0000 7f31977fe640 1 -- 172.21.15.57:0/1333136503 <== mgr.4100 v2:172.21.15.80:6816/2493457677 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f31a00b53a0 con 0x7f3180067880 2024-04-25T23:54:05.200 INFO:teuthology.orchestra.run.smithi057.stderr:instructed osd(s) 6 to deep-scrub 2024-04-25T23:54:05.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.216+0000 7f31a5416640 1 -- 172.21.15.57:0/1333136503 >> v2:172.21.15.80:6816/2493457677 conn(0x7f3180067880 msgr2=0x7f3180069d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:05.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.216+0000 7f31a5416640 1 --2- 172.21.15.57:0/1333136503 >> v2:172.21.15.80:6816/2493457677 conn(0x7f3180067880 0x7f3180069d40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f31a00b1980 tx=0x7f3188009040 comp rx=0 tx=0).stop 2024-04-25T23:54:05.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.216+0000 7f31a5416640 1 -- 172.21.15.57:0/1333136503 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f318c003a10 msgr2=0x7f31a01d18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:05.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.216+0000 7f31a5416640 1 --2- 172.21.15.57:0/1333136503 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f318c003a10 0x7f31a01d18b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f319001c930 tx=0x7f318c005840 comp rx=0 tx=0).stop 2024-04-25T23:54:05.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.216+0000 7f31a5416640 1 -- 172.21.15.57:0/1333136503 shutdown_connections 2024-04-25T23:54:05.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.216+0000 7f31a5416640 1 --2- 172.21.15.57:0/1333136503 >> v2:172.21.15.80:6816/2493457677 conn(0x7f3180067880 0x7f3180069d40 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:05.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.216+0000 7f31a5416640 1 --2- 172.21.15.57:0/1333136503 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f31a01d23f0 0x7f31a0140fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:05.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.216+0000 7f31a5416640 1 --2- 172.21.15.57:0/1333136503 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f31a01d29f0 0x7f31a01d0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:05.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.216+0000 7f31a5416640 1 --2- 172.21.15.57:0/1333136503 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f318c003a10 0x7f31a01d18b0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:05.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.216+0000 7f31a5416640 1 -- 172.21.15.57:0/1333136503 >> 172.21.15.57:0/1333136503 conn(0x7f31a00b58a0 msgr2=0x7f31a01435c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:54:05.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.216+0000 7f31a5416640 1 -- 172.21.15.57:0/1333136503 shutdown_connections 2024-04-25T23:54:05.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.216+0000 7f31a5416640 1 -- 172.21.15.57:0/1333136503 wait complete. 2024-04-25T23:54:05.220 INFO:tasks.ceph:Scrubbing osd.7 2024-04-25T23:54:05.220 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-25T23:54:05.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.388+0000 7f155dc39640 1 Processor -- start 2024-04-25T23:54:05.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.388+0000 7f155dc39640 1 -- start start 2024-04-25T23:54:05.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.388+0000 7f155dc39640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f1558071d60 0x7f1558070de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:05.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.388+0000 7f155dc39640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f155813c690 0x7f155813cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:05.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.388+0000 7f155dc39640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1558142bd0 con 0x7f155806ea70 2024-04-25T23:54:05.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.388+0000 7f155dc39640 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 -- 0x7f1558072140 con 0x7f1558071d60 2024-04-25T23:54:05.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.388+0000 7f155dc39640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f1558142eb0 con 0x7f155813c690 2024-04-25T23:54:05.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.388+0000 7f15577fe640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f1558071d60 0x7f1558070de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:05.374 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.388+0000 7f154ffff640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f155806ea70 0x7f15581433d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:57862/0 (socket says 172.21.15.57:57862) 2024-04-25T23:54:05.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.388+0000 7f154ffff640 1 -- 172.21.15.57:0/3330187351 learned_addr learned my addr 172.21.15.57:0/3330187351 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-04-25T23:54:05.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.388+0000 7f1557fff640 1 --2- 172.21.15.57:0/3330187351 >> v2:172.21.15.80:3300/0 conn(0x7f155813c690 0x7f155813cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:05.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.388+0000 7f15577fe640 1 -- 172.21.15.57:0/3330187351 >> v2:172.21.15.80:3300/0 conn(0x7f155813c690 msgr2=0x7f155813cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:54:05.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.388+0000 7f15577fe640 1 --2- 172.21.15.57:0/3330187351 >> v2:172.21.15.80:3300/0 conn(0x7f155813c690 0x7f155813cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:05.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.388+0000 7f15577fe640 1 -- 172.21.15.57:0/3330187351 >> v1:172.21.15.57:6789/0 conn(0x7f155806ea70 legacy=0x7f15581433d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:05.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.388+0000 7f15577fe640 1 -- 172.21.15.57:0/3330187351 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1558071320 con 0x7f1558071d60 2024-04-25T23:54:05.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.388+0000 7f1556ffd640 1 -- 172.21.15.57:0/3330187351 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1712607952 0 0) 0x7f1558142bd0 con 0x7f155806ea70 2024-04-25T23:54:05.375 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.388+0000 7f1557fff640 1 --2- 172.21.15.57:0/3330187351 >> v2:172.21.15.80:3300/0 conn(0x7f155813c690 0x7f155813cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T23:54:05.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.392+0000 7f15577fe640 1 --2- 172.21.15.57:0/3330187351 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1558071d60 0x7f1558070de0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f15480097c0 tx=0x7f154801c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=933b99fd8f099e58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:54:05.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.392+0000 7f1556ffd640 1 -- 172.21.15.57:0/3330187351 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15480b0070 con 0x7f1558071d60 2024-04-25T23:54:05.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.392+0000 7f1556ffd640 1 -- 172.21.15.57:0/3330187351 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15480ab600 con 0x7f1558071d60 2024-04-25T23:54:05.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.392+0000 7f1556ffd640 1 -- 172.21.15.57:0/3330187351 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15480ab9d0 con 0x7f1558071d60 2024-04-25T23:54:05.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.392+0000 7f155dc39640 1 -- 172.21.15.57:0/3330187351 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1558071d60 msgr2=0x7f1558070de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:05.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.392+0000 7f155dc39640 1 --2- 172.21.15.57:0/3330187351 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1558071d60 0x7f1558070de0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f15480097c0 tx=0x7f154801c440 comp rx=0 tx=0).stop 2024-04-25T23:54:05.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.392+0000 7f155dc39640 1 -- 172.21.15.57:0/3330187351 shutdown_connections 2024-04-25T23:54:05.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.392+0000 7f155dc39640 1 --2- 172.21.15.57:0/3330187351 >> v2:172.21.15.80:3300/0 conn(0x7f155813c690 0x7f155813cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:05.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.392+0000 7f155dc39640 1 --2- 172.21.15.57:0/3330187351 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1558071d60 0x7f1558070de0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:05.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.392+0000 7f155dc39640 1 -- 172.21.15.57:0/3330187351 >> 172.21.15.57:0/3330187351 conn(0x7f15580b8d70 msgr2=0x7f15580b9180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:54:05.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.392+0000 7f155dc39640 1 -- 172.21.15.57:0/3330187351 shutdown_connections 2024-04-25T23:54:05.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.392+0000 7f155dc39640 1 -- 172.21.15.57:0/3330187351 wait complete. 2024-04-25T23:54:05.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.392+0000 7f155dc39640 1 Processor -- start 2024-04-25T23:54:05.378 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.392+0000 7f155dc39640 1 -- start start 2024-04-25T23:54:05.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.396+0000 7f155dc39640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f155806ea70 0x7f15581de8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:05.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.396+0000 7f155dc39640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1558071d60 0x7f1558138c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:05.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.396+0000 7f155dc39640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f155813c690 0x7f155812fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:05.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.396+0000 7f155dc39640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f155814ff10 con 0x7f1558071d60 2024-04-25T23:54:05.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.396+0000 7f155dc39640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f155806f080 con 0x7f155813c690 2024-04-25T23:54:05.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.396+0000 7f155dc39640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f1558150090 con 0x7f155806ea70 2024-04-25T23:54:05.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.396+0000 7f154ffff640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1558071d60 0x7f1558138c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:05.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.396+0000 7f15577fe640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f155806ea70 0x7f15581de8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:05.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.396+0000 7f154ffff640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1558071d60 0x7f1558138c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38114/0 (socket says 172.21.15.57:38114) 2024-04-25T23:54:05.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.396+0000 7f15577fe640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f155806ea70 0x7f15581de8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3301/0 says I am v2:172.21.15.57:59912/0 (socket says 172.21.15.57:59912) 2024-04-25T23:54:05.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.396+0000 7f154ffff640 1 -- 172.21.15.57:0/643725235 learned_addr learned my addr 172.21.15.57:0/643725235 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:54:05.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.396+0000 7f1557fff640 1 --2- 172.21.15.57:0/643725235 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f155813c690 0x7f155812fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:05.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.396+0000 7f154ffff640 1 -- 172.21.15.57:0/643725235 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f155806ea70 msgr2=0x7f15581de8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:05.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.396+0000 7f154ffff640 1 --2- 172.21.15.57:0/643725235 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f155806ea70 0x7f15581de8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:05.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.396+0000 7f154ffff640 1 -- 172.21.15.57:0/643725235 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f155813c690 msgr2=0x7f155812fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:05.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.396+0000 7f154ffff640 1 --2- 172.21.15.57:0/643725235 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f155813c690 0x7f155812fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:05.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.396+0000 7f154ffff640 1 -- 172.21.15.57:0/643725235 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f155813feb0 con 0x7f1558071d60 2024-04-25T23:54:05.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.396+0000 7f154ffff640 1 --2- 172.21.15.57:0/643725235 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1558071d60 0x7f1558138c90 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f154000c520 tx=0x7f154000c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:54:05.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.396+0000 7f1554ff9640 1 -- 172.21.15.57:0/643725235 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1540018070 con 0x7f1558071d60 2024-04-25T23:54:05.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.396+0000 7f1554ff9640 1 -- 172.21.15.57:0/643725235 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15400074e0 con 0x7f1558071d60 2024-04-25T23:54:05.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.396+0000 7f155dc39640 1 -- 172.21.15.57:0/643725235 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1558139e10 con 0x7f1558071d60 2024-04-25T23:54:05.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.396+0000 7f1554ff9640 1 -- 172.21.15.57:0/643725235 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15400049f0 con 0x7f1558071d60 2024-04-25T23:54:05.382 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.396+0000 7f155dc39640 1 -- 172.21.15.57:0/643725235 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1558130700 con 0x7f1558071d60 2024-04-25T23:54:05.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.396+0000 7f155dc39640 1 -- 172.21.15.57:0/643725235 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f155813fc60 con 0x7f1558071d60 2024-04-25T23:54:05.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.396+0000 7f1554ff9640 1 -- 172.21.15.57:0/643725235 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) ==== 87410+0+0 (secure 0 0 0) 0x7f154001b070 con 0x7f1558071d60 2024-04-25T23:54:05.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.396+0000 7f1554ff9640 1 --2- 172.21.15.57:0/643725235 >> v2:172.21.15.80:6816/2493457677 conn(0x7f1538067880 0x7f1538069d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:05.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.396+0000 7f1554ff9640 1 -- 172.21.15.57:0/643725235 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 4525+0+0 (secure 0 0 0) 0x7f1540084ce0 con 0x7f1558071d60 2024-04-25T23:54:05.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.396+0000 7f1554ff9640 1 --2- 172.21.15.57:0/643725235 >> v2:172.21.15.80:6808/2647667231 conn(0x7f153806e5e0 0x7f1538070a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:05.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.396+0000 7f1554ff9640 1 -- 172.21.15.57:0/643725235 --> v2:172.21.15.80:6808/2647667231 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1538071070 con 0x7f153806e5e0 2024-04-25T23:54:05.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.396+0000 7f1554ff9640 1 -- 172.21.15.57:0/643725235 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f1540085170 con 0x7f1558071d60 2024-04-25T23:54:05.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.400+0000 7f15577fe640 1 --2- 172.21.15.57:0/643725235 >> v2:172.21.15.80:6816/2493457677 conn(0x7f1538067880 0x7f1538069d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:05.383 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.400+0000 7f1557fff640 1 --2- 172.21.15.57:0/643725235 >> v2:172.21.15.80:6808/2647667231 conn(0x7f153806e5e0 0x7f1538070a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:05.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.400+0000 7f15577fe640 1 --2- 172.21.15.57:0/643725235 >> v2:172.21.15.80:6816/2493457677 conn(0x7f1538067880 0x7f1538069d40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f15480ad000 tx=0x7f15480a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:54:05.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.400+0000 7f1557fff640 1 --2- 172.21.15.57:0/643725235 >> v2:172.21.15.80:6808/2647667231 conn(0x7f153806e5e0 0x7f1538070a00 crc :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:54:05.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.400+0000 7f1554ff9640 1 -- 172.21.15.57:0/643725235 <== osd.7 v2:172.21.15.80:6808/2647667231 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1538071070 con 0x7f153806e5e0 2024-04-25T23:54:05.399 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.416+0000 7f155dc39640 1 -- 172.21.15.57:0/643725235 --> v2:172.21.15.80:6808/2647667231 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f15580761e0 con 0x7f153806e5e0 2024-04-25T23:54:05.400 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.416+0000 7f1554ff9640 1 -- 172.21.15.57:0/643725235 <== osd.7 v2:172.21.15.80:6808/2647667231 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f15580761e0 con 0x7f153806e5e0 2024-04-25T23:54:05.400 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-04-25T23:54:05.400 INFO:teuthology.orchestra.run.smithi057.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T23:54:05.400 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-04-25T23:54:05.401 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.416+0000 7f155dc39640 1 -- 172.21.15.57:0/643725235 >> v2:172.21.15.80:6808/2647667231 conn(0x7f153806e5e0 msgr2=0x7f1538070a00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:05.401 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.416+0000 7f155dc39640 1 --2- 172.21.15.57:0/643725235 >> v2:172.21.15.80:6808/2647667231 conn(0x7f153806e5e0 0x7f1538070a00 crc :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:05.401 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.416+0000 7f155dc39640 1 -- 172.21.15.57:0/643725235 >> v2:172.21.15.80:6816/2493457677 conn(0x7f1538067880 msgr2=0x7f1538069d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:05.401 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.416+0000 7f155dc39640 1 --2- 172.21.15.57:0/643725235 >> v2:172.21.15.80:6816/2493457677 conn(0x7f1538067880 0x7f1538069d40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f15480ad000 tx=0x7f15480a6000 comp rx=0 tx=0).stop 2024-04-25T23:54:05.401 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.416+0000 7f155dc39640 1 -- 172.21.15.57:0/643725235 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1558071d60 msgr2=0x7f1558138c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:05.401 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.416+0000 7f155dc39640 1 --2- 172.21.15.57:0/643725235 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1558071d60 0x7f1558138c90 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f154000c520 tx=0x7f154000c9e0 comp rx=0 tx=0).stop 2024-04-25T23:54:05.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.416+0000 7f1557fff640 1 -- 172.21.15.57:0/643725235 reap_dead start 2024-04-25T23:54:05.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.416+0000 7f155dc39640 1 -- 172.21.15.57:0/643725235 shutdown_connections 2024-04-25T23:54:05.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.416+0000 7f155dc39640 1 -- 172.21.15.57:0/643725235 >> 172.21.15.57:0/643725235 conn(0x7f15580b8d70 msgr2=0x7f15580b5bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:54:05.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.416+0000 7f155dc39640 1 -- 172.21.15.57:0/643725235 shutdown_connections 2024-04-25T23:54:05.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.416+0000 7f155dc39640 1 -- 172.21.15.57:0/643725235 wait complete. 2024-04-25T23:54:05.417 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-25T23:54:05.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.572+0000 7f2f4eb2b640 1 Processor -- start 2024-04-25T23:54:05.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.572+0000 7f2f4eb2b640 1 -- start start 2024-04-25T23:54:05.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.572+0000 7f2f4eb2b640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f2f48071d00 0x7f2f4806fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:05.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.572+0000 7f2f4eb2b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f2f4813f500 0x7f2f48140760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:05.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.572+0000 7f2f4eb2b640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f480720e0 con 0x7f2f48140340 2024-04-25T23:54:05.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.572+0000 7f2f4eb2b640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f2f4813fe20 con 0x7f2f4813f500 2024-04-25T23:54:05.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.572+0000 7f2f4eb2b640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f2f48072b10 con 0x7f2f48071d00 2024-04-25T23:54:05.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.572+0000 7f2f4c8c7640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f2f48071d00 0x7f2f4806fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:05.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.572+0000 7f2f47fff640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f2f48140340 0x7f2f4813edf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:57866/0 (socket says 172.21.15.57:57866) 2024-04-25T23:54:05.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.572+0000 7f2f47fff640 1 -- 172.21.15.57:0/2136333987 learned_addr learned my addr 172.21.15.57:0/2136333987 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-04-25T23:54:05.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.572+0000 7f2f4c8c7640 1 --2- 172.21.15.57:0/2136333987 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f2f48071d00 0x7f2f4806fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3301/0 says I am v2:172.21.15.57:59922/0 (socket says 172.21.15.57:59922) 2024-04-25T23:54:05.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.572+0000 7f2f4d0c8640 1 --2- 172.21.15.57:0/2136333987 >> v2:172.21.15.80:3300/0 conn(0x7f2f4813f500 0x7f2f48140760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:05.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.572+0000 7f2f477fe640 1 -- 172.21.15.57:0/2136333987 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 884162120 0 0) 0x7f2f3c001120 con 0x7f2f48140340 2024-04-25T23:54:05.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.572+0000 7f2f477fe640 1 -- 172.21.15.57:0/2136333987 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2f30003530 con 0x7f2f48140340 2024-04-25T23:54:05.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.572+0000 7f2f4c8c7640 1 -- 172.21.15.57:0/2136333987 >> v2:172.21.15.80:3300/0 conn(0x7f2f4813f500 msgr2=0x7f2f48140760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:54:05.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.572+0000 7f2f4c8c7640 1 --2- 172.21.15.57:0/2136333987 >> v2:172.21.15.80:3300/0 conn(0x7f2f4813f500 0x7f2f48140760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:05.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.572+0000 7f2f4c8c7640 1 -- 172.21.15.57:0/2136333987 >> v1:172.21.15.57:6789/0 conn(0x7f2f48140340 legacy=0x7f2f4813edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:05.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.572+0000 7f2f4c8c7640 1 -- 172.21.15.57:0/2136333987 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f4813fa40 con 0x7f2f48071d00 2024-04-25T23:54:05.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.576+0000 7f2f4c8c7640 1 --2- 172.21.15.57:0/2136333987 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f2f48071d00 0x7f2f4806fbd0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2f38009d90 tx=0x7f2f3801c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35629df565570386 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:54:05.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.576+0000 7f2f477fe640 1 -- 172.21.15.57:0/2136333987 <== mon.2 v2:172.21.15.57:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f380b2070 con 0x7f2f48071d00 2024-04-25T23:54:05.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.576+0000 7f2f477fe640 1 -- 172.21.15.57:0/2136333987 <== mon.2 v2:172.21.15.57:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f380a5890 con 0x7f2f48071d00 2024-04-25T23:54:05.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.576+0000 7f2f477fe640 1 -- 172.21.15.57:0/2136333987 <== mon.2 v2:172.21.15.57:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f380a5c60 con 0x7f2f48071d00 2024-04-25T23:54:05.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.576+0000 7f2f4eb2b640 1 -- 172.21.15.57:0/2136333987 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f2f48071d00 msgr2=0x7f2f4806fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:05.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.576+0000 7f2f4eb2b640 1 --2- 172.21.15.57:0/2136333987 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f2f48071d00 0x7f2f4806fbd0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2f38009d90 tx=0x7f2f3801c8c0 comp rx=0 tx=0).stop 2024-04-25T23:54:05.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.576+0000 7f2f4eb2b640 1 -- 172.21.15.57:0/2136333987 shutdown_connections 2024-04-25T23:54:05.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.576+0000 7f2f4eb2b640 1 --2- 172.21.15.57:0/2136333987 >> v2:172.21.15.80:3300/0 conn(0x7f2f4813f500 0x7f2f48140760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:05.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.576+0000 7f2f4eb2b640 1 --2- 172.21.15.57:0/2136333987 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f2f48071d00 0x7f2f4806fbd0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:05.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.576+0000 7f2f4eb2b640 1 -- 172.21.15.57:0/2136333987 >> 172.21.15.57:0/2136333987 conn(0x7f2f480b5a70 msgr2=0x7f2f480b5e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:54:05.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.576+0000 7f2f4eb2b640 1 -- 172.21.15.57:0/2136333987 shutdown_connections 2024-04-25T23:54:05.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.576+0000 7f2f4eb2b640 1 -- 172.21.15.57:0/2136333987 wait complete. 2024-04-25T23:54:05.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.580+0000 7f2f4eb2b640 1 Processor -- start 2024-04-25T23:54:05.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.580+0000 7f2f4eb2b640 1 -- start start 2024-04-25T23:54:05.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.580+0000 7f2f4eb2b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2f48071d00 0x7f2f48130490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:05.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.580+0000 7f2f4eb2b640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f4813f500 0x7f2f4812f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:05.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.580+0000 7f2f4eb2b640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f2f48140340 0x7f2f48129a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:05.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.580+0000 7f2f4eb2b640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f2f4814cf00 con 0x7f2f4813f500 2024-04-25T23:54:05.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.580+0000 7f2f4eb2b640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f2f4814cd80 con 0x7f2f48071d00 2024-04-25T23:54:05.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.580+0000 7f2f4eb2b640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f2f4814d080 con 0x7f2f48140340 2024-04-25T23:54:05.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.580+0000 7f2f4d0c8640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f2f48140340 0x7f2f48129a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:05.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.580+0000 7f2f47fff640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f4813f500 0x7f2f4812f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:05.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.580+0000 7f2f4d0c8640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f2f48140340 0x7f2f48129a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3301/0 says I am v2:172.21.15.57:59928/0 (socket says 172.21.15.57:59928) 2024-04-25T23:54:05.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.580+0000 7f2f4d0c8640 1 -- 172.21.15.57:0/492321577 learned_addr learned my addr 172.21.15.57:0/492321577 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:54:05.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.580+0000 7f2f4c8c7640 1 --2- 172.21.15.57:0/492321577 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2f48071d00 0x7f2f48130490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:05.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.580+0000 7f2f4d0c8640 1 -- 172.21.15.57:0/492321577 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2f48071d00 msgr2=0x7f2f48130490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:05.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.580+0000 7f2f4d0c8640 1 --2- 172.21.15.57:0/492321577 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2f48071d00 0x7f2f48130490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:05.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.580+0000 7f2f4d0c8640 1 -- 172.21.15.57:0/492321577 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f4813f500 msgr2=0x7f2f4812f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:05.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.580+0000 7f2f4d0c8640 1 --2- 172.21.15.57:0/492321577 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f4813f500 0x7f2f4812f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:05.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.580+0000 7f2f4d0c8640 1 -- 172.21.15.57:0/492321577 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f481384d0 con 0x7f2f48140340 2024-04-25T23:54:05.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.580+0000 7f2f4d0c8640 1 --2- 172.21.15.57:0/492321577 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f2f48140340 0x7f2f48129a20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2f4000e960 tx=0x7f2f4000ee20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:54:05.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.580+0000 7f2f457fa640 1 -- 172.21.15.57:0/492321577 <== mon.2 v2:172.21.15.57:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f40017070 con 0x7f2f48140340 2024-04-25T23:54:05.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.580+0000 7f2f457fa640 1 -- 172.21.15.57:0/492321577 <== mon.2 v2:172.21.15.57:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f4000c9a0 con 0x7f2f48140340 2024-04-25T23:54:05.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.580+0000 7f2f4eb2b640 1 -- 172.21.15.57:0/492321577 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f4812fa60 con 0x7f2f48140340 2024-04-25T23:54:05.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.580+0000 7f2f457fa640 1 -- 172.21.15.57:0/492321577 <== mon.2 v2:172.21.15.57:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4000cd70 con 0x7f2f48140340 2024-04-25T23:54:05.567 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.580+0000 7f2f4eb2b640 1 -- 172.21.15.57:0/492321577 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f48129f60 con 0x7f2f48140340 2024-04-25T23:54:05.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.580+0000 7f2f4eb2b640 1 -- 172.21.15.57:0/492321577 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f480761e0 con 0x7f2f48140340 2024-04-25T23:54:05.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.584+0000 7f2f457fa640 1 -- 172.21.15.57:0/492321577 <== mon.2 v2:172.21.15.57:3301/0 4 ==== mgrmap(e 5) ==== 87410+0+0 (secure 0 0 0) 0x7f2f4001a070 con 0x7f2f48140340 2024-04-25T23:54:05.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.584+0000 7f2f457fa640 1 --2- 172.21.15.57:0/492321577 >> v2:172.21.15.80:6816/2493457677 conn(0x7f2f34067880 0x7f2f34069d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:05.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.584+0000 7f2f457fa640 1 -- 172.21.15.57:0/492321577 <== mon.2 v2:172.21.15.57:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 4525+0+0 (secure 0 0 0) 0x7f2f40085610 con 0x7f2f48140340 2024-04-25T23:54:05.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.584+0000 7f2f4c8c7640 1 --2- 172.21.15.57:0/492321577 >> v2:172.21.15.80:6816/2493457677 conn(0x7f2f34067880 0x7f2f34069d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:05.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.584+0000 7f2f4c8c7640 1 --2- 172.21.15.57:0/492321577 >> v2:172.21.15.80:6816/2493457677 conn(0x7f2f34067880 0x7f2f34069d40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2f380af000 tx=0x7f2f380a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:54:05.573 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.588+0000 7f2f457fa640 1 -- 172.21.15.57:0/492321577 <== mon.2 v2:172.21.15.57:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2f40056030 con 0x7f2f48140340 2024-04-25T23:54:05.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.820+0000 7f2f4eb2b640 1 -- 172.21.15.57:0/492321577 --> v2:172.21.15.80:6816/2493457677 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f2f4813e850 con 0x7f2f34067880 2024-04-25T23:54:05.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.820+0000 7f2f457fa640 1 -- 172.21.15.57:0/492321577 <== mgr.4100 v2:172.21.15.80:6816/2493457677 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f2f4813e850 con 0x7f2f34067880 2024-04-25T23:54:05.804 INFO:teuthology.orchestra.run.smithi057.stderr:instructed osd(s) 7 to deep-scrub 2024-04-25T23:54:05.806 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.824+0000 7f2f4eb2b640 1 -- 172.21.15.57:0/492321577 >> v2:172.21.15.80:6816/2493457677 conn(0x7f2f34067880 msgr2=0x7f2f34069d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:05.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.824+0000 7f2f4eb2b640 1 --2- 172.21.15.57:0/492321577 >> v2:172.21.15.80:6816/2493457677 conn(0x7f2f34067880 0x7f2f34069d40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2f380af000 tx=0x7f2f380a7070 comp rx=0 tx=0).stop 2024-04-25T23:54:05.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.824+0000 7f2f4eb2b640 1 -- 172.21.15.57:0/492321577 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f2f48140340 msgr2=0x7f2f48129a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:05.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.824+0000 7f2f4eb2b640 1 --2- 172.21.15.57:0/492321577 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f2f48140340 0x7f2f48129a20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2f4000e960 tx=0x7f2f4000ee20 comp rx=0 tx=0).stop 2024-04-25T23:54:05.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.824+0000 7f2f4eb2b640 1 -- 172.21.15.57:0/492321577 shutdown_connections 2024-04-25T23:54:05.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.824+0000 7f2f4eb2b640 1 --2- 172.21.15.57:0/492321577 >> v2:172.21.15.80:6816/2493457677 conn(0x7f2f34067880 0x7f2f34069d40 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:05.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.824+0000 7f2f4eb2b640 1 --2- 172.21.15.57:0/492321577 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f2f48140340 0x7f2f48129a20 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:05.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.824+0000 7f2f4eb2b640 1 --2- 172.21.15.57:0/492321577 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f4813f500 0x7f2f4812f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:05.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.824+0000 7f2f4eb2b640 1 --2- 172.21.15.57:0/492321577 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2f48071d00 0x7f2f48130490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:05.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.824+0000 7f2f4eb2b640 1 -- 172.21.15.57:0/492321577 >> 172.21.15.57:0/492321577 conn(0x7f2f480b5a70 msgr2=0x7f2f480748f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:54:05.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.824+0000 7f2f4eb2b640 1 -- 172.21.15.57:0/492321577 shutdown_connections 2024-04-25T23:54:05.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.824+0000 7f2f4eb2b640 1 -- 172.21.15.57:0/492321577 wait complete. 2024-04-25T23:54:05.827 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T23:54:05.958 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.972+0000 7f73bb180640 1 Processor -- start 2024-04-25T23:54:05.958 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.972+0000 7f73bb180640 1 -- start start 2024-04-25T23:54:05.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.976+0000 7f73bb180640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f73b4071d50 0x7f73b4070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:05.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.976+0000 7f73bb180640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f73b4142740 0x7f73b4146670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:05.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.976+0000 7f73bb180640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73b4072130 con 0x7f73b406ea90 2024-04-25T23:54:05.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.976+0000 7f73bb180640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f73b40adb10 con 0x7f73b4142740 2024-04-25T23:54:05.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.976+0000 7f73bb180640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f73b41461e0 con 0x7f73b4071d50 2024-04-25T23:54:05.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.976+0000 7f73b8f1c640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f73b4071d50 0x7f73b4070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:05.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.976+0000 7f73b8f1c640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f73b4071d50 0x7f73b4070ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3301/0 says I am v2:172.21.15.57:59942/0 (socket says 172.21.15.57:59942) 2024-04-25T23:54:05.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.976+0000 7f73b8f1c640 1 -- 172.21.15.57:0/1608036087 learned_addr learned my addr 172.21.15.57:0/1608036087 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:54:05.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.976+0000 7f73b971d640 1 --2- 172.21.15.57:0/1608036087 >> v2:172.21.15.80:3300/0 conn(0x7f73b4142740 0x7f73b4146670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:05.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.976+0000 7f73b8f1c640 1 -- 172.21.15.57:0/1608036087 >> v2:172.21.15.80:3300/0 conn(0x7f73b4142740 msgr2=0x7f73b4146670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:54:05.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.976+0000 7f73b8f1c640 1 --2- 172.21.15.57:0/1608036087 >> v2:172.21.15.80:3300/0 conn(0x7f73b4142740 0x7f73b4146670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:05.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.976+0000 7f73b8f1c640 1 -- 172.21.15.57:0/1608036087 >> v1:172.21.15.80:6789/0 conn(0x7f73b406ea90 legacy=0x7f73b4142030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:54:05.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.976+0000 7f73b8f1c640 1 -- 172.21.15.57:0/1608036087 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73b4071640 con 0x7f73b4071d50 2024-04-25T23:54:05.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.976+0000 7f73b8f1c640 1 --2- 172.21.15.57:0/1608036087 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f73b4071d50 0x7f73b4070ff0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f73a4009870 tx=0x7f73a401c320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=543461dea53a82c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:54:05.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.976+0000 7f73b37fe640 1 -- 172.21.15.57:0/1608036087 <== mon.2 v2:172.21.15.57:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a40b0070 con 0x7f73b4071d50 2024-04-25T23:54:05.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.976+0000 7f73b37fe640 1 -- 172.21.15.57:0/1608036087 <== mon.2 v2:172.21.15.57:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73a40ab600 con 0x7f73b4071d50 2024-04-25T23:54:05.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.976+0000 7f73b37fe640 1 -- 172.21.15.57:0/1608036087 <== mon.2 v2:172.21.15.57:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a40ab9d0 con 0x7f73b4071d50 2024-04-25T23:54:05.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.976+0000 7f73bb180640 1 -- 172.21.15.57:0/1608036087 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f73b4071d50 msgr2=0x7f73b4070ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:05.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.976+0000 7f73bb180640 1 --2- 172.21.15.57:0/1608036087 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f73b4071d50 0x7f73b4070ff0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f73a4009870 tx=0x7f73a401c320 comp rx=0 tx=0).stop 2024-04-25T23:54:05.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.976+0000 7f73bb180640 1 -- 172.21.15.57:0/1608036087 shutdown_connections 2024-04-25T23:54:05.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.976+0000 7f73bb180640 1 --2- 172.21.15.57:0/1608036087 >> v2:172.21.15.80:3300/0 conn(0x7f73b4142740 0x7f73b4146670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:05.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.976+0000 7f73bb180640 1 --2- 172.21.15.57:0/1608036087 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f73b4071d50 0x7f73b4070ff0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:05.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.976+0000 7f73bb180640 1 -- 172.21.15.57:0/1608036087 >> 172.21.15.57:0/1608036087 conn(0x7f73b40b9110 msgr2=0x7f73b40b9520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:54:05.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.976+0000 7f73bb180640 1 -- 172.21.15.57:0/1608036087 shutdown_connections 2024-04-25T23:54:05.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.976+0000 7f73bb180640 1 -- 172.21.15.57:0/1608036087 wait complete. 2024-04-25T23:54:05.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.980+0000 7f73bb180640 1 Processor -- start 2024-04-25T23:54:05.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.980+0000 7f73bb180640 1 -- start start 2024-04-25T23:54:05.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.980+0000 7f73bb180640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f73b406ea90 0x7f73b4132b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:05.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.980+0000 7f73bb180640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f73b4071d50 0x7f73b4141160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:05.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.980+0000 7f73bb180640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73b4142740 0x7f73b41de8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:05.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.980+0000 7f73bb180640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f73b4150150 con 0x7f73b4142740 2024-04-25T23:54:05.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.980+0000 7f73bb180640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f73b414ffd0 con 0x7f73b406ea90 2024-04-25T23:54:05.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.980+0000 7f73bb180640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7f73b41502d0 con 0x7f73b4071d50 2024-04-25T23:54:05.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.980+0000 7f73b3fff640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f73b4071d50 0x7f73b4141160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:05.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.980+0000 7f73b971d640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73b4142740 0x7f73b41de8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:05.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.980+0000 7f73b3fff640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f73b4071d50 0x7f73b4141160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3301/0 says I am v2:172.21.15.57:59954/0 (socket says 172.21.15.57:59954) 2024-04-25T23:54:05.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.980+0000 7f73b3fff640 1 -- 172.21.15.57:0/4001342212 learned_addr learned my addr 172.21.15.57:0/4001342212 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:54:05.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.980+0000 7f73b971d640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73b4142740 0x7f73b41de8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38146/0 (socket says 172.21.15.57:38146) 2024-04-25T23:54:05.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.980+0000 7f73b8f1c640 1 --2- 172.21.15.57:0/4001342212 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f73b406ea90 0x7f73b4132b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:05.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.980+0000 7f73b3fff640 1 -- 172.21.15.57:0/4001342212 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f73b406ea90 msgr2=0x7f73b4132b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:05.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.980+0000 7f73b3fff640 1 --2- 172.21.15.57:0/4001342212 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f73b406ea90 0x7f73b4132b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:05.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.980+0000 7f73b3fff640 1 -- 172.21.15.57:0/4001342212 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73b4142740 msgr2=0x7f73b41de8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:05.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.980+0000 7f73b3fff640 1 --2- 172.21.15.57:0/4001342212 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73b4142740 0x7f73b41de8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:05.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.980+0000 7f73b3fff640 1 -- 172.21.15.57:0/4001342212 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73b4141830 con 0x7f73b4071d50 2024-04-25T23:54:05.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.980+0000 7f73b3fff640 1 --2- 172.21.15.57:0/4001342212 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f73b4071d50 0x7f73b4141160 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f73a800c7d0 tx=0x7f73a800cc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:54:05.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.980+0000 7f73b17fa640 1 -- 172.21.15.57:0/4001342212 <== mon.2 v2:172.21.15.57:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a8017070 con 0x7f73b4071d50 2024-04-25T23:54:05.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.980+0000 7f73b17fa640 1 -- 172.21.15.57:0/4001342212 <== mon.2 v2:172.21.15.57:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73a8003db0 con 0x7f73b4071d50 2024-04-25T23:54:05.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.980+0000 7f73bb180640 1 -- 172.21.15.57:0/4001342212 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73b4138b50 con 0x7f73b4071d50 2024-04-25T23:54:05.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.980+0000 7f73b17fa640 1 -- 172.21.15.57:0/4001342212 <== mon.2 v2:172.21.15.57:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a80052b0 con 0x7f73b4071d50 2024-04-25T23:54:05.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.980+0000 7f73bb180640 1 -- 172.21.15.57:0/4001342212 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f73b413a890 con 0x7f73b4071d50 2024-04-25T23:54:05.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.980+0000 7f73bb180640 1 -- 172.21.15.57:0/4001342212 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73b40761e0 con 0x7f73b4071d50 2024-04-25T23:54:05.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.984+0000 7f73b17fa640 1 -- 172.21.15.57:0/4001342212 <== mon.2 v2:172.21.15.57:3301/0 4 ==== mgrmap(e 5) ==== 87410+0+0 (secure 0 0 0) 0x7f73a801a070 con 0x7f73b4071d50 2024-04-25T23:54:05.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.984+0000 7f73b17fa640 1 --2- 172.21.15.57:0/4001342212 >> v2:172.21.15.80:6816/2493457677 conn(0x7f73a0067880 0x7f73a0069d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:05.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.984+0000 7f73b17fa640 1 -- 172.21.15.57:0/4001342212 <== mon.2 v2:172.21.15.57:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 4525+0+0 (secure 0 0 0) 0x7f73a8084dd0 con 0x7f73b4071d50 2024-04-25T23:54:05.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.984+0000 7f73b8f1c640 1 --2- 172.21.15.57:0/4001342212 >> v2:172.21.15.80:6816/2493457677 conn(0x7f73a0067880 0x7f73a0069d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:05.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.984+0000 7f73b8f1c640 1 --2- 172.21.15.57:0/4001342212 >> v2:172.21.15.80:6816/2493457677 conn(0x7f73a0067880 0x7f73a0069d40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f73a40ad000 tx=0x7f73a40a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:54:05.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:05.988+0000 7f73b17fa640 1 -- 172.21.15.57:0/4001342212 <== mon.2 v2:172.21.15.57:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f73a80557f0 con 0x7f73b4071d50 2024-04-25T23:54:06.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:06.144+0000 7f73bb180640 1 -- 172.21.15.57:0/4001342212 --> v2:172.21.15.80:6816/2493457677 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f73b40b9690 con 0x7f73a0067880 2024-04-25T23:54:06.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:06.156+0000 7f73b17fa640 1 -- 172.21.15.57:0/4001342212 <== mgr.4100 v2:172.21.15.80:6816/2493457677 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+117572 (secure 0 0 0) 0x7f73b40b9690 con 0x7f73a0067880 2024-04-25T23:54:06.141 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-25T23:54:06.142 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2024-04-25T23:54:06.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:06.160+0000 7f73bb180640 1 -- 172.21.15.57:0/4001342212 >> v2:172.21.15.80:6816/2493457677 conn(0x7f73a0067880 msgr2=0x7f73a0069d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:06.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:06.160+0000 7f73bb180640 1 --2- 172.21.15.57:0/4001342212 >> v2:172.21.15.80:6816/2493457677 conn(0x7f73a0067880 0x7f73a0069d40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f73a40ad000 tx=0x7f73a40a6000 comp rx=0 tx=0).stop 2024-04-25T23:54:06.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:06.160+0000 7f73bb180640 1 -- 172.21.15.57:0/4001342212 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f73b4071d50 msgr2=0x7f73b4141160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:06.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:06.160+0000 7f73bb180640 1 --2- 172.21.15.57:0/4001342212 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f73b4071d50 0x7f73b4141160 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f73a800c7d0 tx=0x7f73a800cc90 comp rx=0 tx=0).stop 2024-04-25T23:54:06.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:06.160+0000 7f73bb180640 1 -- 172.21.15.57:0/4001342212 shutdown_connections 2024-04-25T23:54:06.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:06.160+0000 7f73bb180640 1 --2- 172.21.15.57:0/4001342212 >> v2:172.21.15.80:6816/2493457677 conn(0x7f73a0067880 0x7f73a0069d40 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:06.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:06.160+0000 7f73bb180640 1 --2- 172.21.15.57:0/4001342212 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73b4142740 0x7f73b41de8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:06.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:06.160+0000 7f73bb180640 1 --2- 172.21.15.57:0/4001342212 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7f73b4071d50 0x7f73b4141160 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:06.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:06.160+0000 7f73bb180640 1 --2- 172.21.15.57:0/4001342212 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f73b406ea90 0x7f73b4132b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:06.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:06.160+0000 7f73bb180640 1 -- 172.21.15.57:0/4001342212 >> 172.21.15.57:0/4001342212 conn(0x7f73b40b9110 msgr2=0x7f73b40b5720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:54:06.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:06.160+0000 7f73bb180640 1 -- 172.21.15.57:0/4001342212 shutdown_connections 2024-04-25T23:54:06.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:06.160+0000 7f73bb180640 1 -- 172.21.15.57:0/4001342212 wait complete. 2024-04-25T23:54:06.160 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":169,"stamp":"2024-04-25T23:54:04.920832+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3561427736,"num_objects":878,"num_object_clones":0,"num_object_copies":1756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":878,"num_whiteouts":0,"num_read":2682,"num_read_kb":2682,"num_write":155867,"num_write_kb":5622897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157642,"ondisk_log_size":157642,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":9015900,"kb_used_data":6625536,"kb_used_omap":12,"kb_used_meta":2390259,"kb_avail":745958820,"statfs":{"total":773094113280,"available":763861831680,"internally_reserved":0,"allocated":6784548864,"data_stored":6821938952,"data_compressed":194300696,"data_compressed_allocated":202137600,"data_compressed_original":244916224,"omap_allocated":12751,"internal_metadata":2447625777},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8720,"num_objects":-1,"num_object_clones":0,"num_object_copies":-2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1,"num_whiteouts":0,"num_read":66,"num_read_kb":71,"num_write":17,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003134"},"pg_stats":[{"pgid":"3.1f","version":"20'4490","reported_seq":4532,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:53:58.260328+0000","last_change":"2024-04-25T23:49:04.991824+0000","last_active":"2024-04-25T23:53:58.260328+0000","last_peered":"2024-04-25T23:53:58.260328+0000","last_clean":"2024-04-25T23:53:58.260328+0000","last_became_active":"2024-04-25T23:49:04.991059+0000","last_became_peered":"2024-04-25T23:49:04.991059+0000","last_unstale":"2024-04-25T23:53:58.260328+0000","last_undegraded":"2024-04-25T23:53:58.260328+0000","last_fullsized":"2024-04-25T23:53:58.260328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":4490,"log_dups_size":0,"ondisk_log_size":4490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121634816,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":93,"num_read_kb":81,"num_write":4513,"num_write_kb":218308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"22'5439","reported_seq":5468,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:53:58.200248+0000","last_change":"2024-04-25T23:49:04.991993+0000","last_active":"2024-04-25T23:53:58.200248+0000","last_peered":"2024-04-25T23:53:58.200248+0000","last_clean":"2024-04-25T23:53:58.200248+0000","last_became_active":"2024-04-25T23:49:04.991224+0000","last_became_peered":"2024-04-25T23:49:04.991224+0000","last_unstale":"2024-04-25T23:53:58.200248+0000","last_undegraded":"2024-04-25T23:53:58.200248+0000","last_fullsized":"2024-04-25T23:53:58.200248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":5439,"log_dups_size":0,"ondisk_log_size":5439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:49:45.384039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":45,"num_read_kb":37,"num_write":5439,"num_write_kb":172929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"22'6314","reported_seq":6340,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:53:58.264219+0000","last_change":"2024-04-25T23:49:04.984448+0000","last_active":"2024-04-25T23:53:58.264219+0000","last_peered":"2024-04-25T23:53:58.264219+0000","last_clean":"2024-04-25T23:53:58.264219+0000","last_became_active":"2024-04-25T23:49:04.982204+0000","last_became_peered":"2024-04-25T23:49:04.982204+0000","last_unstale":"2024-04-25T23:53:58.264219+0000","last_undegraded":"2024-04-25T23:53:58.264219+0000","last_fullsized":"2024-04-25T23:53:58.264219+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":6314,"log_dups_size":0,"ondisk_log_size":6314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:23:55.891971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":150994944,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":30,"num_read_kb":85,"num_write":6420,"num_write_kb":204243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"20'3855","reported_seq":3876,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:53:58.263741+0000","last_change":"2024-04-25T23:49:04.980119+0000","last_active":"2024-04-25T23:53:58.263741+0000","last_peered":"2024-04-25T23:53:58.263741+0000","last_clean":"2024-04-25T23:53:58.263741+0000","last_became_active":"2024-04-25T23:49:04.979323+0000","last_became_peered":"2024-04-25T23:49:04.979323+0000","last_unstale":"2024-04-25T23:53:58.263741+0000","last_undegraded":"2024-04-25T23:53:58.263741+0000","last_fullsized":"2024-04-25T23:53:58.263741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":3855,"log_dups_size":0,"ondisk_log_size":3855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:09:46.513536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96535048,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":17,"num_read_kb":12,"num_write":3868,"num_write_kb":155287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"22'4572","reported_seq":4590,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:53:58.284949+0000","last_change":"2024-04-25T23:49:05.002350+0000","last_active":"2024-04-25T23:53:58.284949+0000","last_peered":"2024-04-25T23:53:58.284949+0000","last_clean":"2024-04-25T23:53:58.284949+0000","last_became_active":"2024-04-25T23:49:05.001246+0000","last_became_peered":"2024-04-25T23:49:05.001246+0000","last_unstale":"2024-04-25T23:53:58.284949+0000","last_undegraded":"2024-04-25T23:53:58.284949+0000","last_fullsized":"2024-04-25T23:53:58.284949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":4572,"log_dups_size":0,"ondisk_log_size":4572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":107675648,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4523,"num_write_kb":223556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"22'6960","reported_seq":6977,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:53:58.192089+0000","last_change":"2024-04-25T23:49:04.984321+0000","last_active":"2024-04-25T23:53:58.192089+0000","last_peered":"2024-04-25T23:53:58.192089+0000","last_clean":"2024-04-25T23:53:58.192089+0000","last_became_active":"2024-04-25T23:49:04.983518+0000","last_became_peered":"2024-04-25T23:49:04.983518+0000","last_unstale":"2024-04-25T23:53:58.192089+0000","last_undegraded":"2024-04-25T23:53:58.192089+0000","last_fullsized":"2024-04-25T23:53:58.192089+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":6960,"log_dups_size":0,"ondisk_log_size":6960,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:31:32.872285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":138412032,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6933,"num_write_kb":206597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"22'5766","reported_seq":5784,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:53:58.278374+0000","last_change":"2024-04-25T23:49:04.985251+0000","last_active":"2024-04-25T23:53:58.278374+0000","last_peered":"2024-04-25T23:53:58.278374+0000","last_clean":"2024-04-25T23:53:58.278374+0000","last_became_active":"2024-04-25T23:49:04.984378+0000","last_became_peered":"2024-04-25T23:49:04.984378+0000","last_unstale":"2024-04-25T23:53:58.278374+0000","last_undegraded":"2024-04-25T23:53:58.278374+0000","last_fullsized":"2024-04-25T23:53:58.278374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":5766,"log_dups_size":0,"ondisk_log_size":5766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:43:34.630095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121634816,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5717,"num_write_kb":246662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"21'7903","reported_seq":7922,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:53:58.199845+0000","last_change":"2024-04-25T23:49:04.988246+0000","last_active":"2024-04-25T23:53:58.199845+0000","last_peered":"2024-04-25T23:53:58.199845+0000","last_clean":"2024-04-25T23:53:58.199845+0000","last_became_active":"2024-04-25T23:49:04.987468+0000","last_became_peered":"2024-04-25T23:49:04.987468+0000","last_unstale":"2024-04-25T23:53:58.199845+0000","last_undegraded":"2024-04-25T23:53:58.199845+0000","last_fullsized":"2024-04-25T23:53:58.199845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":7903,"log_dups_size":0,"ondisk_log_size":7903,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:21:42.911815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":130023936,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":7908,"num_write_kb":222727,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"21'5984","reported_seq":6011,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:53:58.261098+0000","last_change":"2024-04-25T23:49:04.989029+0000","last_active":"2024-04-25T23:53:58.261098+0000","last_peered":"2024-04-25T23:53:58.261098+0000","last_clean":"2024-04-25T23:53:58.261098+0000","last_became_active":"2024-04-25T23:49:04.987729+0000","last_became_peered":"2024-04-25T23:49:04.987729+0000","last_unstale":"2024-04-25T23:53:58.261098+0000","last_undegraded":"2024-04-25T23:53:58.261098+0000","last_fullsized":"2024-04-25T23:53:58.261098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":5984,"log_dups_size":0,"ondisk_log_size":5984,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121634816,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":5995,"num_write_kb":201733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"22'3558","reported_seq":3884,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:53:58.264416+0000","last_change":"2024-04-25T23:49:04.986698+0000","last_active":"2024-04-25T23:53:58.264416+0000","last_peered":"2024-04-25T23:53:58.264416+0000","last_clean":"2024-04-25T23:53:58.264416+0000","last_became_active":"2024-04-25T23:49:04.985957+0000","last_became_peered":"2024-04-25T23:49:04.985957+0000","last_unstale":"2024-04-25T23:53:58.264416+0000","last_undegraded":"2024-04-25T23:53:58.264416+0000","last_fullsized":"2024-04-25T23:53:58.264416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":3558,"log_dups_size":0,"ondisk_log_size":3558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:07:54.597105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":77201936,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":1028,"num_read_kb":921,"num_write":3751,"num_write_kb":130317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"21'7645","reported_seq":7663,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:53:58.213108+0000","last_change":"2024-04-25T23:49:04.992705+0000","last_active":"2024-04-25T23:53:58.213108+0000","last_peered":"2024-04-25T23:53:58.213108+0000","last_clean":"2024-04-25T23:53:58.213108+0000","last_became_active":"2024-04-25T23:49:04.990812+0000","last_became_peered":"2024-04-25T23:49:04.990812+0000","last_unstale":"2024-04-25T23:53:58.213108+0000","last_undegraded":"2024-04-25T23:53:58.213108+0000","last_fullsized":"2024-04-25T23:53:58.213108+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":7645,"log_dups_size":0,"ondisk_log_size":7645,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:27:48.304794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":138412032,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7636,"num_write_kb":198278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"22'5032","reported_seq":5066,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:53:58.281717+0000","last_change":"2024-04-25T23:49:04.990899+0000","last_active":"2024-04-25T23:53:58.281717+0000","last_peered":"2024-04-25T23:53:58.281717+0000","last_clean":"2024-04-25T23:53:58.281717+0000","last_became_active":"2024-04-25T23:49:04.988250+0000","last_became_peered":"2024-04-25T23:49:04.988250+0000","last_unstale":"2024-04-25T23:53:58.281717+0000","last_undegraded":"2024-04-25T23:53:58.281717+0000","last_fullsized":"2024-04-25T23:53:58.281717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":5032,"log_dups_size":0,"ondisk_log_size":5032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:53:15.377369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":95029072,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":68,"num_read_kb":57,"num_write":5006,"num_write_kb":140314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":24,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:53:58.263132+0000","last_change":"2024-04-25T23:48:58.911464+0000","last_active":"2024-04-25T23:53:58.263132+0000","last_peered":"2024-04-25T23:53:58.263132+0000","last_clean":"2024-04-25T23:53:58.263132+0000","last_became_active":"2024-04-25T23:48:58.911101+0000","last_became_peered":"2024-04-25T23:48:58.911101+0000","last_unstale":"2024-04-25T23:53:58.263132+0000","last_undegraded":"2024-04-25T23:53:58.263132+0000","last_fullsized":"2024-04-25T23:53:58.263132+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_clean_scrub_stamp":"2024-04-25T23:48:57.875228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:06:28.527623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"22'4733","reported_seq":4753,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:53:58.271541+0000","last_change":"2024-04-25T23:49:04.985258+0000","last_active":"2024-04-25T23:53:58.271541+0000","last_peered":"2024-04-25T23:53:58.271541+0000","last_clean":"2024-04-25T23:53:58.271541+0000","last_became_active":"2024-04-25T23:49:04.984516+0000","last_became_peered":"2024-04-25T23:49:04.984516+0000","last_unstale":"2024-04-25T23:53:58.271541+0000","last_undegraded":"2024-04-25T23:53:58.271541+0000","last_fullsized":"2024-04-25T23:53:58.271541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":4733,"log_dups_size":0,"ondisk_log_size":4733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":89522176,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":1,"num_read_kb":64,"num_write":4706,"num_write_kb":131713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":24,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:53:58.201084+0000","last_change":"2024-04-25T23:48:58.906152+0000","last_active":"2024-04-25T23:53:58.201084+0000","last_peered":"2024-04-25T23:53:58.201084+0000","last_clean":"2024-04-25T23:53:58.201084+0000","last_became_active":"2024-04-25T23:48:58.905805+0000","last_became_peered":"2024-04-25T23:48:58.905805+0000","last_unstale":"2024-04-25T23:53:58.201084+0000","last_undegraded":"2024-04-25T23:53:58.201084+0000","last_fullsized":"2024-04-25T23:53:58.201084+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_clean_scrub_stamp":"2024-04-25T23:48:57.875228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:05:52.310323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"22'3164","reported_seq":3195,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:53:58.274373+0000","last_change":"2024-04-25T23:49:04.978319+0000","last_active":"2024-04-25T23:53:58.274373+0000","last_peered":"2024-04-25T23:53:58.274373+0000","last_clean":"2024-04-25T23:53:58.274373+0000","last_became_active":"2024-04-25T23:49:04.977360+0000","last_became_peered":"2024-04-25T23:49:04.977360+0000","last_unstale":"2024-04-25T23:53:58.274373+0000","last_undegraded":"2024-04-25T23:53:58.274373+0000","last_fullsized":"2024-04-25T23:53:58.274373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":3164,"log_dups_size":0,"ondisk_log_size":3164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:51:25.735100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96731136,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":55,"num_read_kb":103,"num_write":3148,"num_write_kb":156614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":24,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:53:58.213796+0000","last_change":"2024-04-25T23:48:59.122975+0000","last_active":"2024-04-25T23:53:58.213796+0000","last_peered":"2024-04-25T23:53:58.213796+0000","last_clean":"2024-04-25T23:53:58.213796+0000","last_became_active":"2024-04-25T23:48:59.122193+0000","last_became_peered":"2024-04-25T23:48:59.122193+0000","last_unstale":"2024-04-25T23:53:58.213796+0000","last_undegraded":"2024-04-25T23:53:58.213796+0000","last_fullsized":"2024-04-25T23:53:58.213796+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_clean_scrub_stamp":"2024-04-25T23:48:57.875228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:09:25.578965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"21'7247","reported_seq":7265,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:53:58.200832+0000","last_change":"2024-04-25T23:49:05.092246+0000","last_active":"2024-04-25T23:53:58.200832+0000","last_peered":"2024-04-25T23:53:58.200832+0000","last_clean":"2024-04-25T23:53:58.200832+0000","last_became_active":"2024-04-25T23:49:05.091503+0000","last_became_peered":"2024-04-25T23:49:05.091503+0000","last_unstale":"2024-04-25T23:53:58.200832+0000","last_undegraded":"2024-04-25T23:53:58.200832+0000","last_fullsized":"2024-04-25T23:53:58.200832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":7247,"log_dups_size":0,"ondisk_log_size":7247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:41:54.323489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":140460032,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7239,"num_write_kb":212436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:53:58.201487+0000","last_change":"2024-04-25T23:48:59.120607+0000","last_active":"2024-04-25T23:53:58.201487+0000","last_peered":"2024-04-25T23:53:58.201487+0000","last_clean":"2024-04-25T23:53:58.201487+0000","last_became_active":"2024-04-25T23:48:59.119535+0000","last_became_peered":"2024-04-25T23:48:59.119535+0000","last_unstale":"2024-04-25T23:53:58.201487+0000","last_undegraded":"2024-04-25T23:53:58.201487+0000","last_fullsized":"2024-04-25T23:53:58.201487+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_clean_scrub_stamp":"2024-04-25T23:48:57.875228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:40:05.201442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"20'4309","reported_seq":4439,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:53:58.264921+0000","last_change":"2024-04-25T23:49:04.990547+0000","last_active":"2024-04-25T23:53:58.264921+0000","last_peered":"2024-04-25T23:53:58.264921+0000","last_clean":"2024-04-25T23:53:58.264921+0000","last_became_active":"2024-04-25T23:49:04.988031+0000","last_became_peered":"2024-04-25T23:49:04.988031+0000","last_unstale":"2024-04-25T23:53:58.264921+0000","last_undegraded":"2024-04-25T23:53:58.264921+0000","last_fullsized":"2024-04-25T23:53:58.264921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":4309,"log_dups_size":0,"ondisk_log_size":4309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:24:35.524086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100794896,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":148,"num_read_kb":139,"num_write":4335,"num_write_kb":170205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"20'3471","reported_seq":3489,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:53:58.192123+0000","last_change":"2024-04-25T23:49:04.995825+0000","last_active":"2024-04-25T23:53:58.192123+0000","last_peered":"2024-04-25T23:53:58.192123+0000","last_clean":"2024-04-25T23:53:58.192123+0000","last_became_active":"2024-04-25T23:49:04.995190+0000","last_became_peered":"2024-04-25T23:49:04.995190+0000","last_unstale":"2024-04-25T23:53:58.192123+0000","last_undegraded":"2024-04-25T23:53:58.192123+0000","last_fullsized":"2024-04-25T23:53:58.192123+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":3471,"log_dups_size":0,"ondisk_log_size":3471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:23:45.839121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113246208,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3469,"num_write_kb":172802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:53:58.264988+0000","last_change":"2024-04-25T23:48:58.920072+0000","last_active":"2024-04-25T23:53:58.264988+0000","last_peered":"2024-04-25T23:53:58.264988+0000","last_clean":"2024-04-25T23:53:58.264988+0000","last_became_active":"2024-04-25T23:48:58.919554+0000","last_became_peered":"2024-04-25T23:48:58.919554+0000","last_unstale":"2024-04-25T23:53:58.264988+0000","last_undegraded":"2024-04-25T23:53:58.264988+0000","last_fullsized":"2024-04-25T23:53:58.264988+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_clean_scrub_stamp":"2024-04-25T23:48:57.875228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:47:42.230724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"22'3180","reported_seq":3212,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:53:58.268213+0000","last_change":"2024-04-25T23:49:05.092897+0000","last_active":"2024-04-25T23:53:58.268213+0000","last_peered":"2024-04-25T23:53:58.268213+0000","last_clean":"2024-04-25T23:53:58.268213+0000","last_became_active":"2024-04-25T23:49:05.092081+0000","last_became_peered":"2024-04-25T23:49:05.092081+0000","last_unstale":"2024-04-25T23:53:58.268213+0000","last_undegraded":"2024-04-25T23:53:58.268213+0000","last_fullsized":"2024-04-25T23:53:58.268213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":3180,"log_dups_size":0,"ondisk_log_size":3180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:15:07.022713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104857600,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":35,"num_read_kb":153,"num_write":3168,"num_write_kb":148628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":24,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:53:58.205899+0000","last_change":"2024-04-25T23:48:58.910610+0000","last_active":"2024-04-25T23:53:58.205899+0000","last_peered":"2024-04-25T23:53:58.205899+0000","last_clean":"2024-04-25T23:53:58.205899+0000","last_became_active":"2024-04-25T23:48:58.910228+0000","last_became_peered":"2024-04-25T23:48:58.910228+0000","last_unstale":"2024-04-25T23:53:58.205899+0000","last_undegraded":"2024-04-25T23:53:58.205899+0000","last_fullsized":"2024-04-25T23:53:58.205899+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_clean_scrub_stamp":"2024-04-25T23:48:57.875228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:40:43.690806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"22'2867","reported_seq":2887,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:53:58.284491+0000","last_change":"2024-04-25T23:49:04.996197+0000","last_active":"2024-04-25T23:53:58.284491+0000","last_peered":"2024-04-25T23:53:58.284491+0000","last_clean":"2024-04-25T23:53:58.284491+0000","last_became_active":"2024-04-25T23:49:04.995711+0000","last_became_peered":"2024-04-25T23:49:04.995711+0000","last_unstale":"2024-04-25T23:53:58.284491+0000","last_undegraded":"2024-04-25T23:53:58.284491+0000","last_fullsized":"2024-04-25T23:53:58.284491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":2867,"log_dups_size":0,"ondisk_log_size":2867,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":95095128,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":2846,"num_write_kb":142623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":24,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:53:58.212975+0000","last_change":"2024-04-25T23:48:58.913466+0000","last_active":"2024-04-25T23:53:58.212975+0000","last_peered":"2024-04-25T23:53:58.212975+0000","last_clean":"2024-04-25T23:53:58.212975+0000","last_became_active":"2024-04-25T23:48:58.912966+0000","last_became_peered":"2024-04-25T23:48:58.912966+0000","last_unstale":"2024-04-25T23:53:58.212975+0000","last_undegraded":"2024-04-25T23:53:58.212975+0000","last_fullsized":"2024-04-25T23:53:58.212975+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_clean_scrub_stamp":"2024-04-25T23:48:57.875228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:25:56.738346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"21'4357","reported_seq":4380,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:53:58.200493+0000","last_change":"2024-04-25T23:49:04.977221+0000","last_active":"2024-04-25T23:53:58.200493+0000","last_peered":"2024-04-25T23:53:58.200493+0000","last_clean":"2024-04-25T23:53:58.200493+0000","last_became_active":"2024-04-25T23:49:04.976471+0000","last_became_peered":"2024-04-25T23:49:04.976471+0000","last_unstale":"2024-04-25T23:53:58.200493+0000","last_undegraded":"2024-04-25T23:53:58.200493+0000","last_fullsized":"2024-04-25T23:53:58.200493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":4357,"log_dups_size":0,"ondisk_log_size":4357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:16:51.256351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88211456,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":22,"num_read_kb":16,"num_write":4360,"num_write_kb":167041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:53:58.264519+0000","last_change":"2024-04-25T23:48:58.915463+0000","last_active":"2024-04-25T23:53:58.264519+0000","last_peered":"2024-04-25T23:53:58.264519+0000","last_clean":"2024-04-25T23:53:58.264519+0000","last_became_active":"2024-04-25T23:48:58.914379+0000","last_became_peered":"2024-04-25T23:48:58.914379+0000","last_unstale":"2024-04-25T23:53:58.264519+0000","last_undegraded":"2024-04-25T23:53:58.264519+0000","last_fullsized":"2024-04-25T23:53:58.264519+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_clean_scrub_stamp":"2024-04-25T23:48:57.875228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:11:09.852719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":174,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:53:58.213052+0000","last_change":"2024-04-25T23:48:59.124853+0000","last_active":"2024-04-25T23:53:58.213052+0000","last_peered":"2024-04-25T23:53:58.213052+0000","last_clean":"2024-04-25T23:53:58.213052+0000","last_became_active":"2024-04-25T23:48:59.122379+0000","last_became_peered":"2024-04-25T23:48:59.122379+0000","last_unstale":"2024-04-25T23:53:58.213052+0000","last_undegraded":"2024-04-25T23:53:58.213052+0000","last_fullsized":"2024-04-25T23:53:58.213052+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:48:57.875228+0000","last_clean_scrub_stamp":"2024-04-25T23:48:57.875228+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:57:31.186906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"22'3900","reported_seq":3943,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:53:58.279811+0000","last_change":"2024-04-25T23:49:04.980999+0000","last_active":"2024-04-25T23:53:58.279811+0000","last_peered":"2024-04-25T23:53:58.279811+0000","last_clean":"2024-04-25T23:53:58.279811+0000","last_became_active":"2024-04-25T23:49:04.980272+0000","last_became_peered":"2024-04-25T23:49:04.980272+0000","last_unstale":"2024-04-25T23:53:58.279811+0000","last_undegraded":"2024-04-25T23:53:58.279811+0000","last_fullsized":"2024-04-25T23:53:58.279811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":3900,"log_dups_size":0,"ondisk_log_size":3900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:43:10.867606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100794368,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":94,"num_read_kb":77,"num_write":3859,"num_write_kb":201602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"22'7836","reported_seq":7981,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:53:58.318298+0000","last_change":"2024-04-25T23:49:05.094703+0000","last_active":"2024-04-25T23:53:58.318298+0000","last_peered":"2024-04-25T23:53:58.318298+0000","last_clean":"2024-04-25T23:53:58.318298+0000","last_became_active":"2024-04-25T23:49:05.094007+0000","last_became_peered":"2024-04-25T23:49:05.094007+0000","last_unstale":"2024-04-25T23:53:58.318298+0000","last_undegraded":"2024-04-25T23:53:58.318298+0000","last_fullsized":"2024-04-25T23:53:58.318298+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":7836,"log_dups_size":0,"ondisk_log_size":7836,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:33:21.090339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":95027712,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":432,"num_read_kb":386,"num_write":5847,"num_write_kb":183317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"22'4057","reported_seq":4082,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:53:58.263226+0000","last_change":"2024-04-25T23:49:04.983367+0000","last_active":"2024-04-25T23:53:58.263226+0000","last_peered":"2024-04-25T23:53:58.263226+0000","last_clean":"2024-04-25T23:53:58.263226+0000","last_became_active":"2024-04-25T23:49:04.982660+0000","last_became_peered":"2024-04-25T23:49:04.982660+0000","last_unstale":"2024-04-25T23:53:58.263226+0000","last_undegraded":"2024-04-25T23:53:58.263226+0000","last_fullsized":"2024-04-25T23:53:58.263226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":4057,"log_dups_size":0,"ondisk_log_size":4057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:34:08.771639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123396624,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":13,"num_read_kb":11,"num_write":4049,"num_write_kb":164094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"20'3795","reported_seq":3813,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:53:58.201312+0000","last_change":"2024-04-25T23:49:04.991004+0000","last_active":"2024-04-25T23:53:58.201312+0000","last_peered":"2024-04-25T23:53:58.201312+0000","last_clean":"2024-04-25T23:53:58.201312+0000","last_became_active":"2024-04-25T23:49:04.990144+0000","last_became_peered":"2024-04-25T23:49:04.990144+0000","last_unstale":"2024-04-25T23:53:58.201312+0000","last_undegraded":"2024-04-25T23:53:58.201312+0000","last_fullsized":"2024-04-25T23:53:58.201312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":3795,"log_dups_size":0,"ondisk_log_size":3795,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:18:20.437365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113246208,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3792,"num_write_kb":169093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"22'3588","reported_seq":3617,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:53:58.274465+0000","last_change":"2024-04-25T23:49:04.988518+0000","last_active":"2024-04-25T23:53:58.274465+0000","last_peered":"2024-04-25T23:53:58.274465+0000","last_clean":"2024-04-25T23:53:58.274465+0000","last_became_active":"2024-04-25T23:49:04.987663+0000","last_became_peered":"2024-04-25T23:49:04.987663+0000","last_unstale":"2024-04-25T23:53:58.274465+0000","last_undegraded":"2024-04-25T23:53:58.274465+0000","last_fullsized":"2024-04-25T23:53:58.274465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":3588,"log_dups_size":0,"ondisk_log_size":3588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:08:28.379346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112721920,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":45,"num_read_kb":34,"num_write":3560,"num_write_kb":152196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"22'4480","reported_seq":4503,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:53:58.213371+0000","last_change":"2024-04-25T23:49:04.997861+0000","last_active":"2024-04-25T23:53:58.213371+0000","last_peered":"2024-04-25T23:53:58.213371+0000","last_clean":"2024-04-25T23:53:58.213371+0000","last_became_active":"2024-04-25T23:49:04.996781+0000","last_became_peered":"2024-04-25T23:49:04.996781+0000","last_unstale":"2024-04-25T23:53:58.213371+0000","last_undegraded":"2024-04-25T23:53:58.213371+0000","last_fullsized":"2024-04-25T23:53:58.213371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":4480,"log_dups_size":0,"ondisk_log_size":4480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:48:47.865840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":127338152,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":8,"num_read_kb":7,"num_write":4466,"num_write_kb":191484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"22'5774","reported_seq":5891,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:53:58.294707+0000","last_change":"2024-04-25T23:49:04.998785+0000","last_active":"2024-04-25T23:53:58.294707+0000","last_peered":"2024-04-25T23:53:58.294707+0000","last_clean":"2024-04-25T23:53:58.294707+0000","last_became_active":"2024-04-25T23:49:04.997977+0000","last_became_peered":"2024-04-25T23:49:04.997977+0000","last_unstale":"2024-04-25T23:53:58.294707+0000","last_undegraded":"2024-04-25T23:53:58.294707+0000","last_fullsized":"2024-04-25T23:53:58.294707+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":5774,"log_dups_size":0,"ondisk_log_size":5774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:13:53.271710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":116263264,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":327,"num_read_kb":255,"num_write":6043,"num_write_kb":177360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"22'3329","reported_seq":3347,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:53:58.281995+0000","last_change":"2024-04-25T23:49:04.998698+0000","last_active":"2024-04-25T23:53:58.281995+0000","last_peered":"2024-04-25T23:53:58.281995+0000","last_clean":"2024-04-25T23:53:58.281995+0000","last_became_active":"2024-04-25T23:49:04.997986+0000","last_became_peered":"2024-04-25T23:49:04.997986+0000","last_unstale":"2024-04-25T23:53:58.281995+0000","last_undegraded":"2024-04-25T23:53:58.281995+0000","last_fullsized":"2024-04-25T23:53:58.281995+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":3329,"log_dups_size":0,"ondisk_log_size":3329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:59:11.386821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663296,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3277,"num_write_kb":152452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"22'3071","reported_seq":3089,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:53:58.284205+0000","last_change":"2024-04-25T23:49:04.990446+0000","last_active":"2024-04-25T23:53:58.284205+0000","last_peered":"2024-04-25T23:53:58.284205+0000","last_clean":"2024-04-25T23:53:58.284205+0000","last_became_active":"2024-04-25T23:49:04.989709+0000","last_became_peered":"2024-04-25T23:49:04.989709+0000","last_unstale":"2024-04-25T23:53:58.284205+0000","last_undegraded":"2024-04-25T23:53:58.284205+0000","last_fullsized":"2024-04-25T23:53:58.284205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":3071,"log_dups_size":0,"ondisk_log_size":3071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:22:01.360709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3021,"num_write_kb":144132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"20'4577","reported_seq":4603,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:53:58.212978+0000","last_change":"2024-04-25T23:49:04.998536+0000","last_active":"2024-04-25T23:53:58.212978+0000","last_peered":"2024-04-25T23:53:58.212978+0000","last_clean":"2024-04-25T23:53:58.212978+0000","last_became_active":"2024-04-25T23:49:04.997718+0000","last_became_peered":"2024-04-25T23:49:04.997718+0000","last_unstale":"2024-04-25T23:53:58.212978+0000","last_undegraded":"2024-04-25T23:53:58.212978+0000","last_fullsized":"2024-04-25T23:53:58.212978+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":4577,"log_dups_size":0,"ondisk_log_size":4577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:42:48.145557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117440512,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":4584,"num_write_kb":172941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"22'4928","reported_seq":4947,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:53:58.283738+0000","last_change":"2024-04-25T23:49:04.988912+0000","last_active":"2024-04-25T23:53:58.283738+0000","last_peered":"2024-04-25T23:53:58.283738+0000","last_clean":"2024-04-25T23:53:58.283738+0000","last_became_active":"2024-04-25T23:49:04.987367+0000","last_became_peered":"2024-04-25T23:49:04.987367+0000","last_unstale":"2024-04-25T23:53:58.283738+0000","last_undegraded":"2024-04-25T23:53:58.283738+0000","last_fullsized":"2024-04-25T23:53:58.283738+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":4928,"log_dups_size":0,"ondisk_log_size":4928,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":149553152,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4881,"num_write_kb":166152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"22'7344","reported_seq":7372,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:53:58.213135+0000","last_change":"2024-04-25T23:49:04.997739+0000","last_active":"2024-04-25T23:53:58.213135+0000","last_peered":"2024-04-25T23:53:58.213135+0000","last_clean":"2024-04-25T23:53:58.213135+0000","last_became_active":"2024-04-25T23:49:04.996841+0000","last_became_peered":"2024-04-25T23:49:04.996841+0000","last_unstale":"2024-04-25T23:53:58.213135+0000","last_undegraded":"2024-04-25T23:53:58.213135+0000","last_fullsized":"2024-04-25T23:53:58.213135+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:49:03.929576+0000","last_clean_scrub_stamp":"2024-04-25T23:49:03.929576+0000","objects_scrubbed":0,"log_size":7344,"log_dups_size":0,"ondisk_log_size":7344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:08:44.872965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83346448,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":33,"num_read_kb":88,"num_write":7349,"num_write_kb":122645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":3560837368,"num_objects":876,"num_object_clones":0,"num_object_copies":1752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":876,"num_whiteouts":0,"num_read":2566,"num_read_kb":2582,"num_write":155708,"num_write_kb":5620481,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":6782771200,"data_stored":6820324472,"data_compressed":194288578,"data_compressed_allocated":202072064,"data_compressed_original":243892224,"omap_allocated":0,"internal_metadata":0},"log_size":157525,"ondisk_log_size":157525,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":12118,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542207,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":388036,"kb_used_data":298616,"kb_used_omap":1,"kb_used_meta":89406,"kb_avail":93983804,"statfs":{"total":96636764160,"available":96239415296,"internally_reserved":0,"allocated":305782784,"data_stored":306549634,"data_compressed":6553827,"data_compressed_allocated":6823936,"data_compressed_original":7905280,"omap_allocated":1593,"internal_metadata":91552199},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542209,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1212832,"kb_used_data":929044,"kb_used_omap":1,"kb_used_meta":283774,"kb_avail":93159008,"statfs":{"total":96636764160,"available":95394824192,"internally_reserved":0,"allocated":951341056,"data_stored":954367762,"data_compressed":23747576,"data_compressed_allocated":24735744,"data_compressed_original":28565504,"omap_allocated":1595,"internal_metadata":290585029},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542207,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1859000,"kb_used_data":1332336,"kb_used_omap":1,"kb_used_meta":526654,"kb_avail":92512840,"statfs":{"total":96636764160,"available":94733148160,"internally_reserved":0,"allocated":1364312064,"data_stored":1378228858,"data_compressed":53769970,"data_compressed_allocated":55988224,"data_compressed_original":70811648,"omap_allocated":1595,"internal_metadata":539294149},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574911,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1460112,"kb_used_data":1031236,"kb_used_omap":1,"kb_used_meta":428862,"kb_avail":92911728,"statfs":{"total":96636764160,"available":95141609472,"internally_reserved":0,"allocated":1055985664,"data_stored":1063153382,"data_compressed":33533197,"data_compressed_allocated":34906112,"data_compressed_original":43012096,"omap_allocated":1590,"internal_metadata":439155146},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574912,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1418028,"kb_used_data":1062048,"kb_used_omap":1,"kb_used_meta":355966,"kb_avail":92953812,"statfs":{"total":96636764160,"available":95184703488,"internally_reserved":0,"allocated":1087537152,"data_stored":1089959202,"data_compressed":21531874,"data_compressed_allocated":22360064,"data_compressed_original":25559040,"omap_allocated":1593,"internal_metadata":364509639},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574911,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":424764,"kb_used_data":303344,"kb_used_omap":1,"kb_used_meta":121406,"kb_avail":93947076,"statfs":{"total":96636764160,"available":96201805824,"internally_reserved":0,"allocated":310624256,"data_stored":314948974,"data_compressed":11229575,"data_compressed_allocated":11784192,"data_compressed_original":16420864,"omap_allocated":1595,"internal_metadata":124320197},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":861356,"kb_used_data":623392,"kb_used_omap":1,"kb_used_meta":237950,"kb_avail":93510484,"statfs":{"total":96636764160,"available":95754735616,"internally_reserved":0,"allocated":638353408,"data_stored":641447450,"data_compressed":20262986,"data_compressed_allocated":21012480,"data_compressed_original":24612864,"omap_allocated":1595,"internal_metadata":243661253},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574911,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1391772,"kb_used_data":1045520,"kb_used_omap":1,"kb_used_meta":346238,"kb_avail":92980068,"statfs":{"total":96636764160,"available":95211589632,"internally_reserved":0,"allocated":1070612480,"data_stored":1073283690,"data_compressed":23671691,"data_compressed_allocated":24526848,"data_compressed_original":28028928,"omap_allocated":1595,"internal_metadata":354548165},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":6059,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":6059,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1070350336,"data_stored":1072685184,"data_compressed":23665632,"data_compressed_allocated":24494080,"data_compressed_original":27516928,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":638144512,"data_stored":641377872,"data_compressed":20262986,"data_compressed_allocated":21012480,"data_compressed_original":24612864,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":310415360,"data_stored":314879396,"data_compressed":11229575,"data_compressed_allocated":11784192,"data_compressed_original":16420864,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1087328256,"data_stored":1089889624,"data_compressed":21531874,"data_compressed_allocated":22360064,"data_compressed_original":25559040,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1364103168,"data_stored":1378159280,"data_compressed":53769970,"data_compressed_allocated":55988224,"data_compressed_original":70811648,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":951132160,"data_stored":954298184,"data_compressed":23747576,"data_compressed_allocated":24735744,"data_compressed_original":28565504,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":305573888,"data_stored":306480056,"data_compressed":6553827,"data_compressed_allocated":6823936,"data_compressed_original":7905280,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1055723520,"data_stored":1062554876,"data_compressed":33527138,"data_compressed_allocated":34873344,"data_compressed_original":42500096,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T23:54:06.165 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-04-25T23:49:03.929576+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=54, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T23:54:06.165 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-04-25T23:49:03.929576+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=54, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T23:54:06.166 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-04-25T23:49:03.929576+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=54, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T23:54:06.166 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-04-25T23:49:03.929576+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=54, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T23:54:06.166 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-04-25T23:49:03.929576+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=54, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T23:54:06.166 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-04-25T23:49:03.929576+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=54, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T23:54:06.166 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-04-25T23:49:03.929576+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=54, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T23:54:06.166 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-04-25T23:49:03.929576+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=54, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T23:54:06.166 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-04-25T23:49:03.929576+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=54, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T23:54:06.166 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-04-25T23:49:03.929576+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=54, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T23:54:06.166 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-04-25T23:49:03.929576+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=54, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T23:54:06.167 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-04-25T23:49:03.929576+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=54, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T23:54:06.167 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-04-25T23:48:57.875228+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=48, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=54, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T23:54:06.167 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-04-25T23:49:03.929576+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=54, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T23:54:06.167 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-04-25T23:48:57.875228+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=48, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=54, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T23:54:06.167 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-04-25T23:49:03.929576+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=54, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T23:54:06.167 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-04-25T23:48:57.875228+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=48, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=54, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T23:54:06.167 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-04-25T23:49:03.929576+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=54, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T23:54:06.167 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-04-25T23:48:57.875228+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=48, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=54, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T23:54:06.167 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-04-25T23:49:03.929576+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=54, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T23:54:06.167 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-04-25T23:49:03.929576+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=54, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T23:54:06.168 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-04-25T23:48:57.875228+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=48, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=54, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T23:54:06.168 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-04-25T23:49:03.929576+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=54, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T23:54:06.168 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-04-25T23:48:57.875228+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=48, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=54, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T23:54:06.168 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-04-25T23:49:03.929576+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=54, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T23:54:06.168 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-04-25T23:48:57.875228+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=48, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=54, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T23:54:06.168 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-04-25T23:49:03.929576+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=54, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T23:54:06.168 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-04-25T23:48:57.875228+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=48, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=54, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T23:54:06.168 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-25T23:48:57.875228+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=48, tm_sec=57, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=54, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T23:54:06.168 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-04-25T23:49:03.929576+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=54, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T23:54:06.169 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-04-25T23:49:03.929576+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=54, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T23:54:06.169 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-04-25T23:49:03.929576+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=54, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T23:54:06.169 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-04-25T23:49:03.929576+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=54, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T23:54:06.169 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-04-25T23:49:03.929576+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=54, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T23:54:06.169 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-04-25T23:49:03.929576+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=54, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T23:54:06.169 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-04-25T23:49:03.929576+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=54, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T23:54:06.169 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-04-25T23:49:03.929576+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=54, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T23:54:06.169 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-04-25T23:49:03.929576+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=54, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T23:54:06.169 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-04-25T23:49:03.929576+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=54, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T23:54:06.169 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-04-25T23:49:03.929576+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=54, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T23:54:06.170 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-04-25T23:49:03.929576+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=23, tm_min=54, tm_sec=0, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T23:54:06.170 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-25T23:54:26.171 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T23:54:26.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.320+0000 7fd6305bc640 1 Processor -- start 2024-04-25T23:54:26.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.320+0000 7fd6305bc640 1 -- start start 2024-04-25T23:54:26.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.324+0000 7fd6305bc640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd628144180 0x7fd62806fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:26.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.324+0000 7fd6305bc640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fd6281403a0 0x7fd6281449d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:26.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.324+0000 7fd6305bc640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6280ab530 con 0x7fd628144560 2024-04-25T23:54:26.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.324+0000 7fd6305bc640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fd6280a9650 con 0x7fd628144180 2024-04-25T23:54:26.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.324+0000 7fd6305bc640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7fd6280aa260 con 0x7fd6281403a0 2024-04-25T23:54:26.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.324+0000 7fd62db57640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fd628144560 0x7fd62813fc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:60932/0 (socket says 172.21.15.57:60932) 2024-04-25T23:54:26.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.324+0000 7fd62eb59640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fd6281403a0 0x7fd6281449d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:26.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.324+0000 7fd62db57640 1 -- 172.21.15.57:0/2502299938 learned_addr learned my addr 172.21.15.57:0/2502299938 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-04-25T23:54:26.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.324+0000 7fd62e358640 1 --2- 172.21.15.57:0/2502299938 >> v2:172.21.15.80:3300/0 conn(0x7fd628144180 0x7fd62806fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:26.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.324+0000 7fd62eb59640 1 -- 172.21.15.57:0/2502299938 >> v2:172.21.15.80:3300/0 conn(0x7fd628144180 msgr2=0x7fd62806fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T23:54:26.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.324+0000 7fd62eb59640 1 --2- 172.21.15.57:0/2502299938 >> v2:172.21.15.80:3300/0 conn(0x7fd628144180 0x7fd62806fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:26.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.324+0000 7fd62eb59640 1 -- 172.21.15.57:0/2502299938 >> v1:172.21.15.57:6789/0 conn(0x7fd628144560 legacy=0x7fd62813fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:26.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.324+0000 7fd62eb59640 1 -- 172.21.15.57:0/2502299938 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6281419b0 con 0x7fd6281403a0 2024-04-25T23:54:26.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.324+0000 7fd62eb59640 1 --2- 172.21.15.57:0/2502299938 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fd6281403a0 0x7fd6281449d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd618009d90 tx=0x7fd61801c750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac77867e84625a2a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:54:26.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.324+0000 7fd62d356640 1 -- 172.21.15.57:0/2502299938 <== mon.2 v2:172.21.15.57:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6180aa5f0 con 0x7fd6281403a0 2024-04-25T23:54:26.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.324+0000 7fd62d356640 1 -- 172.21.15.57:0/2502299938 <== mon.2 v2:172.21.15.57:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6180af070 con 0x7fd6281403a0 2024-04-25T23:54:26.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.324+0000 7fd62d356640 1 -- 172.21.15.57:0/2502299938 <== mon.2 v2:172.21.15.57:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6180aa9b0 con 0x7fd6281403a0 2024-04-25T23:54:26.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.328+0000 7fd6305bc640 1 -- 172.21.15.57:0/2502299938 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fd6281403a0 msgr2=0x7fd6281449d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:26.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.328+0000 7fd6305bc640 1 --2- 172.21.15.57:0/2502299938 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fd6281403a0 0x7fd6281449d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd618009d90 tx=0x7fd61801c750 comp rx=0 tx=0).stop 2024-04-25T23:54:26.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.328+0000 7fd6305bc640 1 -- 172.21.15.57:0/2502299938 shutdown_connections 2024-04-25T23:54:26.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.328+0000 7fd6305bc640 1 --2- 172.21.15.57:0/2502299938 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fd6281403a0 0x7fd6281449d0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:26.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.328+0000 7fd6305bc640 1 --2- 172.21.15.57:0/2502299938 >> v2:172.21.15.80:3300/0 conn(0x7fd628144180 0x7fd62806fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:26.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.328+0000 7fd6305bc640 1 -- 172.21.15.57:0/2502299938 >> 172.21.15.57:0/2502299938 conn(0x7fd6280b1390 msgr2=0x7fd6280b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:54:26.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.328+0000 7fd6305bc640 1 -- 172.21.15.57:0/2502299938 shutdown_connections 2024-04-25T23:54:26.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.328+0000 7fd6305bc640 1 -- 172.21.15.57:0/2502299938 wait complete. 2024-04-25T23:54:26.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.328+0000 7fd6305bc640 1 Processor -- start 2024-04-25T23:54:26.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.328+0000 7fd6305bc640 1 -- start start 2024-04-25T23:54:26.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.328+0000 7fd6305bc640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd6281403a0 0x7fd6281dc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:26.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.328+0000 7fd6305bc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd628144180 0x7fd628139f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:26.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.328+0000 7fd6305bc640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fd628144560 0x7fd62813a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:26.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.328+0000 7fd6305bc640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fd62814de60 con 0x7fd6281403a0 2024-04-25T23:54:26.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.328+0000 7fd6305bc640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd62814dce0 con 0x7fd628144180 2024-04-25T23:54:26.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.328+0000 7fd6305bc640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7fd62814dfe0 con 0x7fd628144560 2024-04-25T23:54:26.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.328+0000 7fd62eb59640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fd628144560 0x7fd62813a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:26.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.328+0000 7fd62eb59640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fd628144560 0x7fd62813a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3301/0 says I am v2:172.21.15.57:55280/0 (socket says 172.21.15.57:55280) 2024-04-25T23:54:26.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.328+0000 7fd62eb59640 1 -- 172.21.15.57:0/1271019155 learned_addr learned my addr 172.21.15.57:0/1271019155 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:54:26.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.328+0000 7fd62e358640 1 --2- 172.21.15.57:0/1271019155 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd6281403a0 0x7fd6281dc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:26.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.328+0000 7fd62db57640 1 --2- 172.21.15.57:0/1271019155 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd628144180 0x7fd628139f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:26.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.328+0000 7fd62eb59640 1 -- 172.21.15.57:0/1271019155 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd628144180 msgr2=0x7fd628139f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:26.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.328+0000 7fd62eb59640 1 --2- 172.21.15.57:0/1271019155 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd628144180 0x7fd628139f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:26.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.328+0000 7fd62eb59640 1 -- 172.21.15.57:0/1271019155 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd6281403a0 msgr2=0x7fd6281dc0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:26.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.328+0000 7fd62eb59640 1 --2- 172.21.15.57:0/1271019155 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd6281403a0 0x7fd6281dc0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:26.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.328+0000 7fd62eb59640 1 -- 172.21.15.57:0/1271019155 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd628139130 con 0x7fd628144560 2024-04-25T23:54:26.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.328+0000 7fd62eb59640 1 --2- 172.21.15.57:0/1271019155 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fd628144560 0x7fd62813a4b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd6180ac040 tx=0x7fd6180194f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:54:26.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.328+0000 7fd626ffd640 1 -- 172.21.15.57:0/1271019155 <== mon.2 v2:172.21.15.57:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6180c3070 con 0x7fd628144560 2024-04-25T23:54:26.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.328+0000 7fd626ffd640 1 -- 172.21.15.57:0/1271019155 <== mon.2 v2:172.21.15.57:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6180b4b70 con 0x7fd628144560 2024-04-25T23:54:26.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.328+0000 7fd6305bc640 1 -- 172.21.15.57:0/1271019155 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd628134280 con 0x7fd628144560 2024-04-25T23:54:26.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.328+0000 7fd626ffd640 1 -- 172.21.15.57:0/1271019155 <== mon.2 v2:172.21.15.57:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6180be420 con 0x7fd628144560 2024-04-25T23:54:26.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.328+0000 7fd6305bc640 1 -- 172.21.15.57:0/1271019155 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6281dc7f0 con 0x7fd628144560 2024-04-25T23:54:26.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.332+0000 7fd6305bc640 1 -- 172.21.15.57:0/1271019155 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6280761e0 con 0x7fd628144560 2024-04-25T23:54:26.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.332+0000 7fd626ffd640 1 -- 172.21.15.57:0/1271019155 <== mon.2 v2:172.21.15.57:3301/0 4 ==== mgrmap(e 5) ==== 87410+0+0 (secure 0 0 0) 0x7fd6180c6070 con 0x7fd628144560 2024-04-25T23:54:26.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.332+0000 7fd62eb59640 1 -- 172.21.15.57:0/1271019155 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fd628144560 msgr2=0x7fd62813a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-04-25T23:54:26.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.332+0000 7fd62eb59640 1 -- 172.21.15.57:0/1271019155 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fd628144560 msgr2=0x7fd62813a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T23:54:26.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.332+0000 7fd62eb59640 1 --2- 172.21.15.57:0/1271019155 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fd628144560 0x7fd62813a4b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd6180ac040 tx=0x7fd6180194f0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T23:54:26.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.332+0000 7fd62eb59640 1 --2- 172.21.15.57:0/1271019155 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fd628144560 0x7fd62813a4b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd6180ac040 tx=0x7fd6180194f0 comp rx=0 tx=0).stop 2024-04-25T23:54:26.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.332+0000 7fd626ffd640 1 --2- 172.21.15.57:0/1271019155 >> v2:172.21.15.80:6816/2493457677 conn(0x7fd614067880 0x7fd614069d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:26.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.332+0000 7fd626ffd640 1 -- 172.21.15.57:0/1271019155 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fd628144560 msgr2=0x7fd62813a4b0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T23:54:26.318 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.332+0000 7fd626ffd640 1 --2- 172.21.15.57:0/1271019155 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fd628144560 0x7fd62813a4b0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:26.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.332+0000 7fd626ffd640 1 --2- 172.21.15.57:0/1271019155 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd628144180 0x7fd61406c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:26.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.332+0000 7fd626ffd640 1 --2- 172.21.15.57:0/1271019155 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fd61406cc00 0x7fd61406f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:26.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.332+0000 7fd626ffd640 1 --2- 172.21.15.57:0/1271019155 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd6281403a0 0x7fd614071b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:26.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.332+0000 7fd626ffd640 1 -- 172.21.15.57:0/1271019155 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fd6140724a0 con 0x7fd6281403a0 2024-04-25T23:54:26.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.332+0000 7fd626ffd640 1 -- 172.21.15.57:0/1271019155 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd614072620 con 0x7fd628144180 2024-04-25T23:54:26.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.332+0000 7fd626ffd640 1 -- 172.21.15.57:0/1271019155 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7fd6140727f0 con 0x7fd61406cc00 2024-04-25T23:54:26.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.332+0000 7fd62e358640 1 --2- 172.21.15.57:0/1271019155 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd6281403a0 0x7fd614071b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:26.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.332+0000 7fd62eb59640 1 --2- 172.21.15.57:0/1271019155 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fd61406cc00 0x7fd61406f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:26.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.332+0000 7fd62e358640 1 --2- 172.21.15.57:0/1271019155 >> v2:172.21.15.80:6816/2493457677 conn(0x7fd614067880 0x7fd614069d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:26.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.332+0000 7fd62db57640 1 --2- 172.21.15.57:0/1271019155 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd628144180 0x7fd61406c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:26.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.332+0000 7fd62eb59640 1 -- 172.21.15.57:0/1271019155 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd628144180 msgr2=0x7fd61406c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:26.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.332+0000 7fd62eb59640 1 --2- 172.21.15.57:0/1271019155 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd628144180 0x7fd61406c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:26.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.332+0000 7fd62eb59640 1 -- 172.21.15.57:0/1271019155 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd6281403a0 msgr2=0x7fd614071b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:26.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.332+0000 7fd62eb59640 1 --2- 172.21.15.57:0/1271019155 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd6281403a0 0x7fd614071b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:26.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.332+0000 7fd62eb59640 1 -- 172.21.15.57:0/1271019155 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_subscribe({config=0+,mgrmap=0+,monmap=2+,osdmap=0}) -- 0x7fd6140729f0 con 0x7fd61406cc00 2024-04-25T23:54:26.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.332+0000 7fd62eb59640 1 -- 172.21.15.57:0/1271019155 --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6280761e0 con 0x7fd61406cc00 2024-04-25T23:54:26.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.332+0000 7fd62e358640 1 --2- 172.21.15.57:0/1271019155 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd6281403a0 0x7fd614071b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T23:54:26.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.332+0000 7fd62eb59640 1 --2- 172.21.15.57:0/1271019155 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fd61406cc00 0x7fd61406f090 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd61811e730 tx=0x7fd6180a7000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:54:26.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.332+0000 7fd62e358640 1 --2- 172.21.15.57:0/1271019155 >> v2:172.21.15.80:6816/2493457677 conn(0x7fd614067880 0x7fd614069d40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd620006e90 tx=0x7fd620008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:54:26.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.336+0000 7fd626ffd640 1 -- 172.21.15.57:0/1271019155 <== mon.2 v2:172.21.15.57:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd618124070 con 0x7fd61406cc00 2024-04-25T23:54:26.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.336+0000 7fd626ffd640 1 -- 172.21.15.57:0/1271019155 <== mon.2 v2:172.21.15.57:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6180af070 con 0x7fd61406cc00 2024-04-25T23:54:26.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.340+0000 7fd626ffd640 1 -- 172.21.15.57:0/1271019155 <== mon.2 v2:172.21.15.57:3301/0 3 ==== mgrmap(e 5) ==== 87410+0+0 (secure 0 0 0) 0x7fd6180ec420 con 0x7fd61406cc00 2024-04-25T23:54:26.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.340+0000 7fd626ffd640 1 -- 172.21.15.57:0/1271019155 <== mon.2 v2:172.21.15.57:3301/0 4 ==== osd_map(22..22 src has 1..22) ==== 4525+0+0 (secure 0 0 0) 0x7fd6180ea150 con 0x7fd61406cc00 2024-04-25T23:54:26.326 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.340+0000 7fd626ffd640 1 -- 172.21.15.57:0/1271019155 <== mon.2 v2:172.21.15.57:3301/0 5 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd6180b9070 con 0x7fd61406cc00 2024-04-25T23:54:26.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.500+0000 7fd6305bc640 1 -- 172.21.15.57:0/1271019155 --> v2:172.21.15.80:6816/2493457677 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd6281dc9c0 con 0x7fd614067880 2024-04-25T23:54:26.493 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.508+0000 7fd626ffd640 1 -- 172.21.15.57:0/1271019155 <== mgr.4100 v2:172.21.15.80:6816/2493457677 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+118042 (secure 0 0 0) 0x7fd6281dc9c0 con 0x7fd614067880 2024-04-25T23:54:26.495 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-04-25T23:54:26.496 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2024-04-25T23:54:26.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.512+0000 7fd6305bc640 1 -- 172.21.15.57:0/1271019155 >> v2:172.21.15.80:6816/2493457677 conn(0x7fd614067880 msgr2=0x7fd614069d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:26.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.512+0000 7fd6305bc640 1 --2- 172.21.15.57:0/1271019155 >> v2:172.21.15.80:6816/2493457677 conn(0x7fd614067880 0x7fd614069d40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd620006e90 tx=0x7fd620008040 comp rx=0 tx=0).stop 2024-04-25T23:54:26.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.512+0000 7fd6305bc640 1 -- 172.21.15.57:0/1271019155 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fd61406cc00 msgr2=0x7fd61406f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:26.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.512+0000 7fd6305bc640 1 --2- 172.21.15.57:0/1271019155 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fd61406cc00 0x7fd61406f090 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd61811e730 tx=0x7fd6180a7000 comp rx=0 tx=0).stop 2024-04-25T23:54:26.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.512+0000 7fd6305bc640 1 -- 172.21.15.57:0/1271019155 shutdown_connections 2024-04-25T23:54:26.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.512+0000 7fd6305bc640 1 --2- 172.21.15.57:0/1271019155 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd628144180 0x7fd61406c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:26.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.512+0000 7fd6305bc640 1 --2- 172.21.15.57:0/1271019155 >> v2:172.21.15.80:6816/2493457677 conn(0x7fd614067880 0x7fd614069d40 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:26.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.512+0000 7fd6305bc640 1 --2- 172.21.15.57:0/1271019155 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd6281403a0 0x7fd614071b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:26.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.512+0000 7fd6305bc640 1 --2- 172.21.15.57:0/1271019155 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7fd61406cc00 0x7fd61406f090 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:26.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.512+0000 7fd6305bc640 1 -- 172.21.15.57:0/1271019155 >> 172.21.15.57:0/1271019155 conn(0x7fd6280b1390 msgr2=0x7fd62813c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:54:26.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.512+0000 7fd6305bc640 1 -- 172.21.15.57:0/1271019155 shutdown_connections 2024-04-25T23:54:26.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.512+0000 7fd6305bc640 1 -- 172.21.15.57:0/1271019155 wait complete. 2024-04-25T23:54:26.512 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":179,"stamp":"2024-04-25T23:54:24.927177+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3561427736,"num_objects":878,"num_object_clones":0,"num_object_copies":1756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":878,"num_whiteouts":0,"num_read":2682,"num_read_kb":2682,"num_write":155867,"num_write_kb":5622897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158517,"ondisk_log_size":158517,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":9024192,"kb_used_data":6625536,"kb_used_omap":12,"kb_used_meta":2398579,"kb_avail":745950528,"statfs":{"total":773094113280,"available":763853340672,"internally_reserved":0,"allocated":6784548864,"data_stored":6821938952,"data_compressed":194300696,"data_compressed_allocated":202137600,"data_compressed_original":244916224,"omap_allocated":12751,"internal_metadata":2456145457},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003745"},"pg_stats":[{"pgid":"3.1f","version":"22'4519","reported_seq":4570,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:54:05.907303+0000","last_change":"2024-04-25T23:54:05.907372+0000","last_active":"2024-04-25T23:54:05.907303+0000","last_peered":"2024-04-25T23:54:05.907303+0000","last_clean":"2024-04-25T23:54:05.907303+0000","last_became_active":"2024-04-25T23:49:04.991059+0000","last_became_peered":"2024-04-25T23:49:04.991059+0000","last_unstale":"2024-04-25T23:54:05.907303+0000","last_undegraded":"2024-04-25T23:54:05.907303+0000","last_fullsized":"2024-04-25T23:54:05.907303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'4519","last_scrub_stamp":"2024-04-25T23:54:05.907166+0000","last_deep_scrub":"22'4519","last_deep_scrub_stamp":"2024-04-25T23:54:05.907166+0000","last_clean_scrub_stamp":"2024-04-25T23:54:05.907166+0000","objects_scrubbed":29,"log_size":4519,"log_dups_size":0,"ondisk_log_size":4519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:20:51.096697+0000","scrub_duration":363,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121634816,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":93,"num_read_kb":81,"num_write":4513,"num_write_kb":218308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"22'5465","reported_seq":5504,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:54:08.853323+0000","last_change":"2024-04-25T23:54:08.853393+0000","last_active":"2024-04-25T23:54:08.853323+0000","last_peered":"2024-04-25T23:54:08.853323+0000","last_clean":"2024-04-25T23:54:08.853323+0000","last_became_active":"2024-04-25T23:49:04.991224+0000","last_became_peered":"2024-04-25T23:49:04.991224+0000","last_unstale":"2024-04-25T23:54:08.853323+0000","last_undegraded":"2024-04-25T23:54:08.853323+0000","last_fullsized":"2024-04-25T23:54:08.853323+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'5465","last_scrub_stamp":"2024-04-25T23:54:08.853151+0000","last_deep_scrub":"22'5465","last_deep_scrub_stamp":"2024-04-25T23:54:08.853151+0000","last_clean_scrub_stamp":"2024-04-25T23:54:08.853151+0000","objects_scrubbed":26,"log_size":5465,"log_dups_size":0,"ondisk_log_size":5465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:25:53.599686+0000","scrub_duration":403,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":45,"num_read_kb":37,"num_write":5439,"num_write_kb":172929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"22'6351","reported_seq":6388,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:54:08.217430+0000","last_change":"2024-04-25T23:54:08.217537+0000","last_active":"2024-04-25T23:54:08.217430+0000","last_peered":"2024-04-25T23:54:08.217430+0000","last_clean":"2024-04-25T23:54:08.217430+0000","last_became_active":"2024-04-25T23:49:04.982204+0000","last_became_peered":"2024-04-25T23:49:04.982204+0000","last_unstale":"2024-04-25T23:54:08.217430+0000","last_undegraded":"2024-04-25T23:54:08.217430+0000","last_fullsized":"2024-04-25T23:54:08.217430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'6351","last_scrub_stamp":"2024-04-25T23:54:08.217337+0000","last_deep_scrub":"22'6351","last_deep_scrub_stamp":"2024-04-25T23:54:08.217337+0000","last_clean_scrub_stamp":"2024-04-25T23:54:08.217337+0000","objects_scrubbed":37,"log_size":6351,"log_dups_size":0,"ondisk_log_size":6351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:02:55.976808+0000","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":150994944,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":30,"num_read_kb":85,"num_write":6420,"num_write_kb":204243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"22'3880","reported_seq":3912,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:54:07.027963+0000","last_change":"2024-04-25T23:54:07.027963+0000","last_active":"2024-04-25T23:54:07.027963+0000","last_peered":"2024-04-25T23:54:07.027963+0000","last_clean":"2024-04-25T23:54:07.027963+0000","last_became_active":"2024-04-25T23:49:04.979323+0000","last_became_peered":"2024-04-25T23:49:04.979323+0000","last_unstale":"2024-04-25T23:54:07.027963+0000","last_undegraded":"2024-04-25T23:54:07.027963+0000","last_fullsized":"2024-04-25T23:54:07.027963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'3880","last_scrub_stamp":"2024-04-25T23:54:07.027764+0000","last_deep_scrub":"22'3880","last_deep_scrub_stamp":"2024-04-25T23:54:07.027764+0000","last_clean_scrub_stamp":"2024-04-25T23:54:07.027764+0000","objects_scrubbed":25,"log_size":3880,"log_dups_size":0,"ondisk_log_size":3880,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:05:58.130212+0000","scrub_duration":379,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96535048,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":17,"num_read_kb":12,"num_write":3868,"num_write_kb":155287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"22'4598","reported_seq":4626,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:54:07.960429+0000","last_change":"2024-04-25T23:54:07.960473+0000","last_active":"2024-04-25T23:54:07.960429+0000","last_peered":"2024-04-25T23:54:07.960429+0000","last_clean":"2024-04-25T23:54:07.960429+0000","last_became_active":"2024-04-25T23:49:05.001246+0000","last_became_peered":"2024-04-25T23:49:05.001246+0000","last_unstale":"2024-04-25T23:54:07.960429+0000","last_undegraded":"2024-04-25T23:54:07.960429+0000","last_fullsized":"2024-04-25T23:54:07.960429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'4598","last_scrub_stamp":"2024-04-25T23:54:07.960319+0000","last_deep_scrub":"22'4598","last_deep_scrub_stamp":"2024-04-25T23:54:07.960319+0000","last_clean_scrub_stamp":"2024-04-25T23:54:07.960319+0000","objects_scrubbed":26,"log_size":4598,"log_dups_size":0,"ondisk_log_size":4598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:24:43.285189+0000","scrub_duration":419,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":107675648,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4523,"num_write_kb":223556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"22'6993","reported_seq":7021,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:54:05.490987+0000","last_change":"2024-04-25T23:54:05.490987+0000","last_active":"2024-04-25T23:54:05.490987+0000","last_peered":"2024-04-25T23:54:05.490987+0000","last_clean":"2024-04-25T23:54:05.490987+0000","last_became_active":"2024-04-25T23:49:04.983518+0000","last_became_peered":"2024-04-25T23:49:04.983518+0000","last_unstale":"2024-04-25T23:54:05.490987+0000","last_undegraded":"2024-04-25T23:54:05.490987+0000","last_fullsized":"2024-04-25T23:54:05.490987+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'6993","last_scrub_stamp":"2024-04-25T23:54:05.490557+0000","last_deep_scrub":"22'6993","last_deep_scrub_stamp":"2024-04-25T23:54:05.490557+0000","last_clean_scrub_stamp":"2024-04-25T23:54:05.490557+0000","objects_scrubbed":33,"log_size":6993,"log_dups_size":0,"ondisk_log_size":6993,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:40:08.454798+0000","scrub_duration":543,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":138412032,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6933,"num_write_kb":206597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"22'5795","reported_seq":5823,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:54:04.901190+0000","last_change":"2024-04-25T23:54:04.901233+0000","last_active":"2024-04-25T23:54:04.901190+0000","last_peered":"2024-04-25T23:54:04.901190+0000","last_clean":"2024-04-25T23:54:04.901190+0000","last_became_active":"2024-04-25T23:49:04.984378+0000","last_became_peered":"2024-04-25T23:49:04.984378+0000","last_unstale":"2024-04-25T23:54:04.901190+0000","last_undegraded":"2024-04-25T23:54:04.901190+0000","last_fullsized":"2024-04-25T23:54:04.901190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'5795","last_scrub_stamp":"2024-04-25T23:54:04.901104+0000","last_deep_scrub":"22'5795","last_deep_scrub_stamp":"2024-04-25T23:54:04.901104+0000","last_clean_scrub_stamp":"2024-04-25T23:54:04.901104+0000","objects_scrubbed":29,"log_size":5795,"log_dups_size":0,"ondisk_log_size":5795,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:23:03.012580+0000","scrub_duration":383,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121634816,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5717,"num_write_kb":246662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"22'7935","reported_seq":7964,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:54:06.971725+0000","last_change":"2024-04-25T23:54:06.971798+0000","last_active":"2024-04-25T23:54:06.971725+0000","last_peered":"2024-04-25T23:54:06.971725+0000","last_clean":"2024-04-25T23:54:06.971725+0000","last_became_active":"2024-04-25T23:49:04.987468+0000","last_became_peered":"2024-04-25T23:49:04.987468+0000","last_unstale":"2024-04-25T23:54:06.971725+0000","last_undegraded":"2024-04-25T23:54:06.971725+0000","last_fullsized":"2024-04-25T23:54:06.971725+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'7935","last_scrub_stamp":"2024-04-25T23:54:06.971610+0000","last_deep_scrub":"22'7935","last_deep_scrub_stamp":"2024-04-25T23:54:06.971610+0000","last_clean_scrub_stamp":"2024-04-25T23:54:06.971610+0000","objects_scrubbed":32,"log_size":7935,"log_dups_size":0,"ondisk_log_size":7935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:45:33.886551+0000","scrub_duration":527,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":130023936,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":7908,"num_write_kb":222727,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"22'6013","reported_seq":6049,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:54:04.995301+0000","last_change":"2024-04-25T23:54:04.995353+0000","last_active":"2024-04-25T23:54:04.995301+0000","last_peered":"2024-04-25T23:54:04.995301+0000","last_clean":"2024-04-25T23:54:04.995301+0000","last_became_active":"2024-04-25T23:49:04.987729+0000","last_became_peered":"2024-04-25T23:49:04.987729+0000","last_unstale":"2024-04-25T23:54:04.995301+0000","last_undegraded":"2024-04-25T23:54:04.995301+0000","last_fullsized":"2024-04-25T23:54:04.995301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'6013","last_scrub_stamp":"2024-04-25T23:54:04.995222+0000","last_deep_scrub":"22'6013","last_deep_scrub_stamp":"2024-04-25T23:54:04.995222+0000","last_clean_scrub_stamp":"2024-04-25T23:54:04.995222+0000","objects_scrubbed":29,"log_size":6013,"log_dups_size":0,"ondisk_log_size":6013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:25:23.987024+0000","scrub_duration":431,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121634816,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":5995,"num_write_kb":201733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"22'3579","reported_seq":3914,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:54:08.949910+0000","last_change":"2024-04-25T23:54:08.950059+0000","last_active":"2024-04-25T23:54:08.949910+0000","last_peered":"2024-04-25T23:54:08.949910+0000","last_clean":"2024-04-25T23:54:08.949910+0000","last_became_active":"2024-04-25T23:49:04.985957+0000","last_became_peered":"2024-04-25T23:49:04.985957+0000","last_unstale":"2024-04-25T23:54:08.949910+0000","last_undegraded":"2024-04-25T23:54:08.949910+0000","last_fullsized":"2024-04-25T23:54:08.949910+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'3579","last_scrub_stamp":"2024-04-25T23:54:08.949685+0000","last_deep_scrub":"22'3579","last_deep_scrub_stamp":"2024-04-25T23:54:08.949685+0000","last_clean_scrub_stamp":"2024-04-25T23:54:08.949685+0000","objects_scrubbed":21,"log_size":3579,"log_dups_size":0,"ondisk_log_size":3579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:26:07.490768+0000","scrub_duration":299,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":77201936,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":1028,"num_read_kb":921,"num_write":3751,"num_write_kb":130317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"22'7678","reported_seq":7706,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:54:10.087681+0000","last_change":"2024-04-25T23:54:10.087760+0000","last_active":"2024-04-25T23:54:10.087681+0000","last_peered":"2024-04-25T23:54:10.087681+0000","last_clean":"2024-04-25T23:54:10.087681+0000","last_became_active":"2024-04-25T23:49:04.990812+0000","last_became_peered":"2024-04-25T23:49:04.990812+0000","last_unstale":"2024-04-25T23:54:10.087681+0000","last_undegraded":"2024-04-25T23:54:10.087681+0000","last_fullsized":"2024-04-25T23:54:10.087681+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'7678","last_scrub_stamp":"2024-04-25T23:54:10.087547+0000","last_deep_scrub":"22'7678","last_deep_scrub_stamp":"2024-04-25T23:54:10.087547+0000","last_clean_scrub_stamp":"2024-04-25T23:54:10.087547+0000","objects_scrubbed":33,"log_size":7678,"log_dups_size":0,"ondisk_log_size":7678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:55:15.014473+0000","scrub_duration":631,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":138412032,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7636,"num_write_kb":198278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"22'5055","reported_seq":5098,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:54:06.342737+0000","last_change":"2024-04-25T23:54:06.342840+0000","last_active":"2024-04-25T23:54:06.342737+0000","last_peered":"2024-04-25T23:54:06.342737+0000","last_clean":"2024-04-25T23:54:06.342737+0000","last_became_active":"2024-04-25T23:49:04.988250+0000","last_became_peered":"2024-04-25T23:49:04.988250+0000","last_unstale":"2024-04-25T23:54:06.342737+0000","last_undegraded":"2024-04-25T23:54:06.342737+0000","last_fullsized":"2024-04-25T23:54:06.342737+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'5055","last_scrub_stamp":"2024-04-25T23:54:06.342642+0000","last_deep_scrub":"22'5055","last_deep_scrub_stamp":"2024-04-25T23:54:06.342642+0000","last_clean_scrub_stamp":"2024-04-25T23:54:06.342642+0000","objects_scrubbed":23,"log_size":5055,"log_dups_size":0,"ondisk_log_size":5055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:12:26.857154+0000","scrub_duration":387,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":95029072,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":68,"num_read_kb":57,"num_write":5006,"num_write_kb":140314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":33,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:54:06.038291+0000","last_change":"2024-04-25T23:54:06.038377+0000","last_active":"2024-04-25T23:54:06.038291+0000","last_peered":"2024-04-25T23:54:06.038291+0000","last_clean":"2024-04-25T23:54:06.038291+0000","last_became_active":"2024-04-25T23:48:58.911101+0000","last_became_peered":"2024-04-25T23:48:58.911101+0000","last_unstale":"2024-04-25T23:54:06.038291+0000","last_undegraded":"2024-04-25T23:54:06.038291+0000","last_fullsized":"2024-04-25T23:54:06.038291+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:54:06.038044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:54:06.038044+0000","last_clean_scrub_stamp":"2024-04-25T23:54:06.038044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:08:49.779814+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"22'4755","reported_seq":4783,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:54:02.777645+0000","last_change":"2024-04-25T23:54:02.777717+0000","last_active":"2024-04-25T23:54:02.777645+0000","last_peered":"2024-04-25T23:54:02.777645+0000","last_clean":"2024-04-25T23:54:02.777645+0000","last_became_active":"2024-04-25T23:49:04.984516+0000","last_became_peered":"2024-04-25T23:49:04.984516+0000","last_unstale":"2024-04-25T23:54:02.777645+0000","last_undegraded":"2024-04-25T23:54:02.777645+0000","last_fullsized":"2024-04-25T23:54:02.777645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'4755","last_scrub_stamp":"2024-04-25T23:54:02.777556+0000","last_deep_scrub":"22'4755","last_deep_scrub_stamp":"2024-04-25T23:54:02.777556+0000","last_clean_scrub_stamp":"2024-04-25T23:54:02.777556+0000","objects_scrubbed":22,"log_size":4755,"log_dups_size":0,"ondisk_log_size":4755,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:05.435457+0000","scrub_duration":259,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":89522176,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":1,"num_read_kb":64,"num_write":4706,"num_write_kb":131713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":33,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:54:02.440810+0000","last_change":"2024-04-25T23:54:02.440860+0000","last_active":"2024-04-25T23:54:02.440810+0000","last_peered":"2024-04-25T23:54:02.440810+0000","last_clean":"2024-04-25T23:54:02.440810+0000","last_became_active":"2024-04-25T23:48:58.905805+0000","last_became_peered":"2024-04-25T23:48:58.905805+0000","last_unstale":"2024-04-25T23:54:02.440810+0000","last_undegraded":"2024-04-25T23:54:02.440810+0000","last_fullsized":"2024-04-25T23:54:02.440810+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:54:02.440645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:54:02.440645+0000","last_clean_scrub_stamp":"2024-04-25T23:54:02.440645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:49:36.268657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"22'3189","reported_seq":3230,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:54:04.756364+0000","last_change":"2024-04-25T23:54:04.756412+0000","last_active":"2024-04-25T23:54:04.756364+0000","last_peered":"2024-04-25T23:54:04.756364+0000","last_clean":"2024-04-25T23:54:04.756364+0000","last_became_active":"2024-04-25T23:49:04.977360+0000","last_became_peered":"2024-04-25T23:49:04.977360+0000","last_unstale":"2024-04-25T23:54:04.756364+0000","last_undegraded":"2024-04-25T23:54:04.756364+0000","last_fullsized":"2024-04-25T23:54:04.756364+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'3189","last_scrub_stamp":"2024-04-25T23:54:04.756218+0000","last_deep_scrub":"22'3189","last_deep_scrub_stamp":"2024-04-25T23:54:04.756218+0000","last_clean_scrub_stamp":"2024-04-25T23:54:04.756218+0000","objects_scrubbed":25,"log_size":3189,"log_dups_size":0,"ondisk_log_size":3189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:33:21.622545+0000","scrub_duration":279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96731136,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":55,"num_read_kb":103,"num_write":3148,"num_write_kb":156614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":33,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:54:07.458337+0000","last_change":"2024-04-25T23:54:07.458410+0000","last_active":"2024-04-25T23:54:07.458337+0000","last_peered":"2024-04-25T23:54:07.458337+0000","last_clean":"2024-04-25T23:54:07.458337+0000","last_became_active":"2024-04-25T23:48:59.122193+0000","last_became_peered":"2024-04-25T23:48:59.122193+0000","last_unstale":"2024-04-25T23:54:07.458337+0000","last_undegraded":"2024-04-25T23:54:07.458337+0000","last_fullsized":"2024-04-25T23:54:07.458337+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:54:07.458127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:54:07.458127+0000","last_clean_scrub_stamp":"2024-04-25T23:54:07.458127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:21:25.905547+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"22'7280","reported_seq":7308,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:54:05.946502+0000","last_change":"2024-04-25T23:54:05.946576+0000","last_active":"2024-04-25T23:54:05.946502+0000","last_peered":"2024-04-25T23:54:05.946502+0000","last_clean":"2024-04-25T23:54:05.946502+0000","last_became_active":"2024-04-25T23:49:05.091503+0000","last_became_peered":"2024-04-25T23:49:05.091503+0000","last_unstale":"2024-04-25T23:54:05.946502+0000","last_undegraded":"2024-04-25T23:54:05.946502+0000","last_fullsized":"2024-04-25T23:54:05.946502+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'7280","last_scrub_stamp":"2024-04-25T23:54:05.946388+0000","last_deep_scrub":"22'7280","last_deep_scrub_stamp":"2024-04-25T23:54:05.946388+0000","last_clean_scrub_stamp":"2024-04-25T23:54:05.946388+0000","objects_scrubbed":34,"log_size":7280,"log_dups_size":0,"ondisk_log_size":7280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:25:58.849656+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":140460032,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7239,"num_write_kb":212436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":33,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:54:03.483113+0000","last_change":"2024-04-25T23:54:03.483153+0000","last_active":"2024-04-25T23:54:03.483113+0000","last_peered":"2024-04-25T23:54:03.483113+0000","last_clean":"2024-04-25T23:54:03.483113+0000","last_became_active":"2024-04-25T23:48:59.119535+0000","last_became_peered":"2024-04-25T23:48:59.119535+0000","last_unstale":"2024-04-25T23:54:03.483113+0000","last_undegraded":"2024-04-25T23:54:03.483113+0000","last_fullsized":"2024-04-25T23:54:03.483113+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:54:03.482968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:54:03.482968+0000","last_clean_scrub_stamp":"2024-04-25T23:54:03.482968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:00.205598+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"22'4334","reported_seq":4474,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:54:10.971375+0000","last_change":"2024-04-25T23:54:10.971421+0000","last_active":"2024-04-25T23:54:10.971375+0000","last_peered":"2024-04-25T23:54:10.971375+0000","last_clean":"2024-04-25T23:54:10.971375+0000","last_became_active":"2024-04-25T23:49:04.988031+0000","last_became_peered":"2024-04-25T23:49:04.988031+0000","last_unstale":"2024-04-25T23:54:10.971375+0000","last_undegraded":"2024-04-25T23:54:10.971375+0000","last_fullsized":"2024-04-25T23:54:10.971375+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'4334","last_scrub_stamp":"2024-04-25T23:54:10.971253+0000","last_deep_scrub":"22'4334","last_deep_scrub_stamp":"2024-04-25T23:54:10.971253+0000","last_clean_scrub_stamp":"2024-04-25T23:54:10.971253+0000","objects_scrubbed":25,"log_size":4334,"log_dups_size":0,"ondisk_log_size":4334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:29:38.349838+0000","scrub_duration":391,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100794896,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":148,"num_read_kb":139,"num_write":4335,"num_write_kb":170205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"22'3498","reported_seq":3526,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:54:07.378434+0000","last_change":"2024-04-25T23:54:07.378523+0000","last_active":"2024-04-25T23:54:07.378434+0000","last_peered":"2024-04-25T23:54:07.378434+0000","last_clean":"2024-04-25T23:54:07.378434+0000","last_became_active":"2024-04-25T23:49:04.995190+0000","last_became_peered":"2024-04-25T23:49:04.995190+0000","last_unstale":"2024-04-25T23:54:07.378434+0000","last_undegraded":"2024-04-25T23:54:07.378434+0000","last_fullsized":"2024-04-25T23:54:07.378434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'3498","last_scrub_stamp":"2024-04-25T23:54:07.378255+0000","last_deep_scrub":"22'3498","last_deep_scrub_stamp":"2024-04-25T23:54:07.378255+0000","last_clean_scrub_stamp":"2024-04-25T23:54:07.378255+0000","objects_scrubbed":27,"log_size":3498,"log_dups_size":0,"ondisk_log_size":3498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:40:00.750104+0000","scrub_duration":395,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113246208,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3469,"num_write_kb":172802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":33,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:54:05.659674+0000","last_change":"2024-04-25T23:54:05.659730+0000","last_active":"2024-04-25T23:54:05.659674+0000","last_peered":"2024-04-25T23:54:05.659674+0000","last_clean":"2024-04-25T23:54:05.659674+0000","last_became_active":"2024-04-25T23:48:58.919554+0000","last_became_peered":"2024-04-25T23:48:58.919554+0000","last_unstale":"2024-04-25T23:54:05.659674+0000","last_undegraded":"2024-04-25T23:54:05.659674+0000","last_fullsized":"2024-04-25T23:54:05.659674+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:54:05.659494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:54:05.659494+0000","last_clean_scrub_stamp":"2024-04-25T23:54:05.659494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:42:39.209396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"22'3206","reported_seq":3248,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:54:06.836709+0000","last_change":"2024-04-25T23:54:06.836740+0000","last_active":"2024-04-25T23:54:06.836709+0000","last_peered":"2024-04-25T23:54:06.836709+0000","last_clean":"2024-04-25T23:54:06.836709+0000","last_became_active":"2024-04-25T23:49:05.092081+0000","last_became_peered":"2024-04-25T23:49:05.092081+0000","last_unstale":"2024-04-25T23:54:06.836709+0000","last_undegraded":"2024-04-25T23:54:06.836709+0000","last_fullsized":"2024-04-25T23:54:06.836709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'3206","last_scrub_stamp":"2024-04-25T23:54:06.836638+0000","last_deep_scrub":"22'3206","last_deep_scrub_stamp":"2024-04-25T23:54:06.836638+0000","last_clean_scrub_stamp":"2024-04-25T23:54:06.836638+0000","objects_scrubbed":26,"log_size":3206,"log_dups_size":0,"ondisk_log_size":3206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:14:23.251899+0000","scrub_duration":307,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104857600,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":35,"num_read_kb":153,"num_write":3168,"num_write_kb":148628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":33,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:54:03.325969+0000","last_change":"2024-04-25T23:54:03.326019+0000","last_active":"2024-04-25T23:54:03.325969+0000","last_peered":"2024-04-25T23:54:03.325969+0000","last_clean":"2024-04-25T23:54:03.325969+0000","last_became_active":"2024-04-25T23:48:58.910228+0000","last_became_peered":"2024-04-25T23:48:58.910228+0000","last_unstale":"2024-04-25T23:54:03.325969+0000","last_undegraded":"2024-04-25T23:54:03.325969+0000","last_fullsized":"2024-04-25T23:54:03.325969+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:54:03.325777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:54:03.325777+0000","last_clean_scrub_stamp":"2024-04-25T23:54:03.325777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:36:26.861819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"22'2891","reported_seq":2919,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:54:06.900607+0000","last_change":"2024-04-25T23:54:06.900721+0000","last_active":"2024-04-25T23:54:06.900607+0000","last_peered":"2024-04-25T23:54:06.900607+0000","last_clean":"2024-04-25T23:54:06.900607+0000","last_became_active":"2024-04-25T23:49:04.995711+0000","last_became_peered":"2024-04-25T23:49:04.995711+0000","last_unstale":"2024-04-25T23:54:06.900607+0000","last_undegraded":"2024-04-25T23:54:06.900607+0000","last_fullsized":"2024-04-25T23:54:06.900607+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'2891","last_scrub_stamp":"2024-04-25T23:54:06.900497+0000","last_deep_scrub":"22'2891","last_deep_scrub_stamp":"2024-04-25T23:54:06.900497+0000","last_clean_scrub_stamp":"2024-04-25T23:54:06.900497+0000","objects_scrubbed":24,"log_size":2891,"log_dups_size":0,"ondisk_log_size":2891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:23:41.471547+0000","scrub_duration":327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":95095128,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":2846,"num_write_kb":142623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":33,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:54:06.447488+0000","last_change":"2024-04-25T23:54:06.447542+0000","last_active":"2024-04-25T23:54:06.447488+0000","last_peered":"2024-04-25T23:54:06.447488+0000","last_clean":"2024-04-25T23:54:06.447488+0000","last_became_active":"2024-04-25T23:48:58.912966+0000","last_became_peered":"2024-04-25T23:48:58.912966+0000","last_unstale":"2024-04-25T23:54:06.447488+0000","last_undegraded":"2024-04-25T23:54:06.447488+0000","last_fullsized":"2024-04-25T23:54:06.447488+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:54:06.447319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:54:06.447319+0000","last_clean_scrub_stamp":"2024-04-25T23:54:06.447319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:19:18.632199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"22'4379","reported_seq":4411,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:54:03.798187+0000","last_change":"2024-04-25T23:54:03.798344+0000","last_active":"2024-04-25T23:54:03.798187+0000","last_peered":"2024-04-25T23:54:03.798187+0000","last_clean":"2024-04-25T23:54:03.798187+0000","last_became_active":"2024-04-25T23:49:04.976471+0000","last_became_peered":"2024-04-25T23:49:04.976471+0000","last_unstale":"2024-04-25T23:54:03.798187+0000","last_undegraded":"2024-04-25T23:54:03.798187+0000","last_fullsized":"2024-04-25T23:54:03.798187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'4379","last_scrub_stamp":"2024-04-25T23:54:03.798066+0000","last_deep_scrub":"22'4379","last_deep_scrub_stamp":"2024-04-25T23:54:03.798066+0000","last_clean_scrub_stamp":"2024-04-25T23:54:03.798066+0000","objects_scrubbed":22,"log_size":4379,"log_dups_size":0,"ondisk_log_size":4379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:16:39.053228+0000","scrub_duration":339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88211456,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":22,"num_read_kb":16,"num_write":4360,"num_write_kb":167041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":33,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:54:04.688913+0000","last_change":"2024-04-25T23:54:04.688970+0000","last_active":"2024-04-25T23:54:04.688913+0000","last_peered":"2024-04-25T23:54:04.688913+0000","last_clean":"2024-04-25T23:54:04.688913+0000","last_became_active":"2024-04-25T23:48:58.914379+0000","last_became_peered":"2024-04-25T23:48:58.914379+0000","last_unstale":"2024-04-25T23:54:04.688913+0000","last_undegraded":"2024-04-25T23:54:04.688913+0000","last_fullsized":"2024-04-25T23:54:04.688913+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T23:54:04.688789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T23:54:04.688789+0000","last_clean_scrub_stamp":"2024-04-25T23:54:04.688789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:24:10.817799+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"22'119","reported_seq":185,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:54:08.442201+0000","last_change":"2024-04-25T23:54:08.442339+0000","last_active":"2024-04-25T23:54:08.442201+0000","last_peered":"2024-04-25T23:54:08.442201+0000","last_clean":"2024-04-25T23:54:08.442201+0000","last_became_active":"2024-04-25T23:48:59.122379+0000","last_became_peered":"2024-04-25T23:48:59.122379+0000","last_unstale":"2024-04-25T23:54:08.442201+0000","last_undegraded":"2024-04-25T23:54:08.442201+0000","last_fullsized":"2024-04-25T23:54:08.442201+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'119","last_scrub_stamp":"2024-04-25T23:54:08.441948+0000","last_deep_scrub":"22'119","last_deep_scrub_stamp":"2024-04-25T23:54:08.441948+0000","last_clean_scrub_stamp":"2024-04-25T23:54:08.441948+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:17:24.678171+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"22'3925","reported_seq":3978,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:54:05.719055+0000","last_change":"2024-04-25T23:54:05.719104+0000","last_active":"2024-04-25T23:54:05.719055+0000","last_peered":"2024-04-25T23:54:05.719055+0000","last_clean":"2024-04-25T23:54:05.719055+0000","last_became_active":"2024-04-25T23:49:04.980272+0000","last_became_peered":"2024-04-25T23:49:04.980272+0000","last_unstale":"2024-04-25T23:54:05.719055+0000","last_undegraded":"2024-04-25T23:54:05.719055+0000","last_fullsized":"2024-04-25T23:54:05.719055+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'3925","last_scrub_stamp":"2024-04-25T23:54:05.718943+0000","last_deep_scrub":"22'3925","last_deep_scrub_stamp":"2024-04-25T23:54:05.718943+0000","last_clean_scrub_stamp":"2024-04-25T23:54:05.718943+0000","objects_scrubbed":25,"log_size":3925,"log_dups_size":0,"ondisk_log_size":3925,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:16:54.389943+0000","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100794368,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":94,"num_read_kb":77,"num_write":3859,"num_write_kb":201602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"22'7859","reported_seq":8014,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:54:08.374723+0000","last_change":"2024-04-25T23:54:08.374865+0000","last_active":"2024-04-25T23:54:08.374723+0000","last_peered":"2024-04-25T23:54:08.374723+0000","last_clean":"2024-04-25T23:54:08.374723+0000","last_became_active":"2024-04-25T23:49:05.094007+0000","last_became_peered":"2024-04-25T23:49:05.094007+0000","last_unstale":"2024-04-25T23:54:08.374723+0000","last_undegraded":"2024-04-25T23:54:08.374723+0000","last_fullsized":"2024-04-25T23:54:08.374723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'7859","last_scrub_stamp":"2024-04-25T23:54:08.374585+0000","last_deep_scrub":"22'7859","last_deep_scrub_stamp":"2024-04-25T23:54:08.374585+0000","last_clean_scrub_stamp":"2024-04-25T23:54:08.374585+0000","objects_scrubbed":24,"log_size":7859,"log_dups_size":0,"ondisk_log_size":7859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:48:36.009741+0000","scrub_duration":403,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":95027712,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":432,"num_read_kb":386,"num_write":5847,"num_write_kb":183317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"22'4087","reported_seq":4122,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:54:07.480622+0000","last_change":"2024-04-25T23:54:07.480723+0000","last_active":"2024-04-25T23:54:07.480622+0000","last_peered":"2024-04-25T23:54:07.480622+0000","last_clean":"2024-04-25T23:54:07.480622+0000","last_became_active":"2024-04-25T23:49:04.982660+0000","last_became_peered":"2024-04-25T23:49:04.982660+0000","last_unstale":"2024-04-25T23:54:07.480622+0000","last_undegraded":"2024-04-25T23:54:07.480622+0000","last_fullsized":"2024-04-25T23:54:07.480622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'4087","last_scrub_stamp":"2024-04-25T23:54:07.480465+0000","last_deep_scrub":"22'4087","last_deep_scrub_stamp":"2024-04-25T23:54:07.480465+0000","last_clean_scrub_stamp":"2024-04-25T23:54:07.480465+0000","objects_scrubbed":31,"log_size":4087,"log_dups_size":0,"ondisk_log_size":4087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:19:02.219108+0000","scrub_duration":451,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123396624,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":13,"num_read_kb":11,"num_write":4049,"num_write_kb":164094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"22'3822","reported_seq":3850,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:54:07.855477+0000","last_change":"2024-04-25T23:54:07.855543+0000","last_active":"2024-04-25T23:54:07.855477+0000","last_peered":"2024-04-25T23:54:07.855477+0000","last_clean":"2024-04-25T23:54:07.855477+0000","last_became_active":"2024-04-25T23:49:04.990144+0000","last_became_peered":"2024-04-25T23:49:04.990144+0000","last_unstale":"2024-04-25T23:54:07.855477+0000","last_undegraded":"2024-04-25T23:54:07.855477+0000","last_fullsized":"2024-04-25T23:54:07.855477+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'3822","last_scrub_stamp":"2024-04-25T23:54:07.855329+0000","last_deep_scrub":"22'3822","last_deep_scrub_stamp":"2024-04-25T23:54:07.855329+0000","last_clean_scrub_stamp":"2024-04-25T23:54:07.855329+0000","objects_scrubbed":27,"log_size":3822,"log_dups_size":0,"ondisk_log_size":3822,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:55:26.944151+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113246208,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3792,"num_write_kb":169093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"22'3617","reported_seq":3656,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:54:10.060598+0000","last_change":"2024-04-25T23:54:10.060767+0000","last_active":"2024-04-25T23:54:10.060598+0000","last_peered":"2024-04-25T23:54:10.060598+0000","last_clean":"2024-04-25T23:54:10.060598+0000","last_became_active":"2024-04-25T23:49:04.987663+0000","last_became_peered":"2024-04-25T23:49:04.987663+0000","last_unstale":"2024-04-25T23:54:10.060598+0000","last_undegraded":"2024-04-25T23:54:10.060598+0000","last_fullsized":"2024-04-25T23:54:10.060598+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'3617","last_scrub_stamp":"2024-04-25T23:54:10.060367+0000","last_deep_scrub":"22'3617","last_deep_scrub_stamp":"2024-04-25T23:54:10.060367+0000","last_clean_scrub_stamp":"2024-04-25T23:54:10.060367+0000","objects_scrubbed":29,"log_size":3617,"log_dups_size":0,"ondisk_log_size":3617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:53:18.842878+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112721920,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":45,"num_read_kb":34,"num_write":3560,"num_write_kb":152196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"22'4511","reported_seq":4544,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:54:11.878344+0000","last_change":"2024-04-25T23:54:11.878488+0000","last_active":"2024-04-25T23:54:11.878344+0000","last_peered":"2024-04-25T23:54:11.878344+0000","last_clean":"2024-04-25T23:54:11.878344+0000","last_became_active":"2024-04-25T23:49:04.996781+0000","last_became_peered":"2024-04-25T23:49:04.996781+0000","last_unstale":"2024-04-25T23:54:11.878344+0000","last_undegraded":"2024-04-25T23:54:11.878344+0000","last_fullsized":"2024-04-25T23:54:11.878344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'4511","last_scrub_stamp":"2024-04-25T23:54:11.878210+0000","last_deep_scrub":"22'4511","last_deep_scrub_stamp":"2024-04-25T23:54:11.878210+0000","last_clean_scrub_stamp":"2024-04-25T23:54:11.878210+0000","objects_scrubbed":31,"log_size":4511,"log_dups_size":0,"ondisk_log_size":4511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:51:27.850817+0000","scrub_duration":479,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":127338152,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":8,"num_read_kb":7,"num_write":4466,"num_write_kb":191484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"22'5804","reported_seq":5931,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:54:14.932728+0000","last_change":"2024-04-25T23:54:14.932896+0000","last_active":"2024-04-25T23:54:14.932728+0000","last_peered":"2024-04-25T23:54:14.932728+0000","last_clean":"2024-04-25T23:54:14.932728+0000","last_became_active":"2024-04-25T23:49:04.997977+0000","last_became_peered":"2024-04-25T23:49:04.997977+0000","last_unstale":"2024-04-25T23:54:14.932728+0000","last_undegraded":"2024-04-25T23:54:14.932728+0000","last_fullsized":"2024-04-25T23:54:14.932728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'5804","last_scrub_stamp":"2024-04-25T23:54:14.932473+0000","last_deep_scrub":"22'5804","last_deep_scrub_stamp":"2024-04-25T23:54:14.932473+0000","last_clean_scrub_stamp":"2024-04-25T23:54:14.932473+0000","objects_scrubbed":30,"log_size":5804,"log_dups_size":0,"ondisk_log_size":5804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:40:31.875372+0000","scrub_duration":507,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":116263264,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":327,"num_read_kb":255,"num_write":6043,"num_write_kb":177360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"22'3353","reported_seq":3380,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:54:13.745938+0000","last_change":"2024-04-25T23:54:13.746053+0000","last_active":"2024-04-25T23:54:13.745938+0000","last_peered":"2024-04-25T23:54:13.745938+0000","last_clean":"2024-04-25T23:54:13.745938+0000","last_became_active":"2024-04-25T23:49:04.997986+0000","last_became_peered":"2024-04-25T23:49:04.997986+0000","last_unstale":"2024-04-25T23:54:13.745938+0000","last_undegraded":"2024-04-25T23:54:13.745938+0000","last_fullsized":"2024-04-25T23:54:13.745938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'3353","last_scrub_stamp":"2024-04-25T23:54:13.745801+0000","last_deep_scrub":"22'3353","last_deep_scrub_stamp":"2024-04-25T23:54:13.745801+0000","last_clean_scrub_stamp":"2024-04-25T23:54:13.745801+0000","objects_scrubbed":24,"log_size":3353,"log_dups_size":0,"ondisk_log_size":3353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:01:00.734818+0000","scrub_duration":347,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663296,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3277,"num_write_kb":152452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"22'3091","reported_seq":3118,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:54:08.285935+0000","last_change":"2024-04-25T23:54:08.286151+0000","last_active":"2024-04-25T23:54:08.285935+0000","last_peered":"2024-04-25T23:54:08.285935+0000","last_clean":"2024-04-25T23:54:08.285935+0000","last_became_active":"2024-04-25T23:49:04.989709+0000","last_became_peered":"2024-04-25T23:49:04.989709+0000","last_unstale":"2024-04-25T23:54:08.285935+0000","last_undegraded":"2024-04-25T23:54:08.285935+0000","last_fullsized":"2024-04-25T23:54:08.285935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'3091","last_scrub_stamp":"2024-04-25T23:54:08.285777+0000","last_deep_scrub":"22'3091","last_deep_scrub_stamp":"2024-04-25T23:54:08.285777+0000","last_clean_scrub_stamp":"2024-04-25T23:54:08.285777+0000","objects_scrubbed":20,"log_size":3091,"log_dups_size":0,"ondisk_log_size":3091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:19:25.588352+0000","scrub_duration":287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3021,"num_write_kb":144132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"22'4605","reported_seq":4641,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:54:12.823307+0000","last_change":"2024-04-25T23:54:12.823381+0000","last_active":"2024-04-25T23:54:12.823307+0000","last_peered":"2024-04-25T23:54:12.823307+0000","last_clean":"2024-04-25T23:54:12.823307+0000","last_became_active":"2024-04-25T23:49:04.997718+0000","last_became_peered":"2024-04-25T23:49:04.997718+0000","last_unstale":"2024-04-25T23:54:12.823307+0000","last_undegraded":"2024-04-25T23:54:12.823307+0000","last_fullsized":"2024-04-25T23:54:12.823307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'4605","last_scrub_stamp":"2024-04-25T23:54:12.823143+0000","last_deep_scrub":"22'4605","last_deep_scrub_stamp":"2024-04-25T23:54:12.823143+0000","last_clean_scrub_stamp":"2024-04-25T23:54:12.823143+0000","objects_scrubbed":28,"log_size":4605,"log_dups_size":0,"ondisk_log_size":4605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:03:42.259549+0000","scrub_duration":467,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117440512,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":4584,"num_write_kb":172941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"22'4964","reported_seq":4992,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:54:04.096595+0000","last_change":"2024-04-25T23:54:04.096693+0000","last_active":"2024-04-25T23:54:04.096595+0000","last_peered":"2024-04-25T23:54:04.096595+0000","last_clean":"2024-04-25T23:54:04.096595+0000","last_became_active":"2024-04-25T23:49:04.987367+0000","last_became_peered":"2024-04-25T23:49:04.987367+0000","last_unstale":"2024-04-25T23:54:04.096595+0000","last_undegraded":"2024-04-25T23:54:04.096595+0000","last_fullsized":"2024-04-25T23:54:04.096595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'4964","last_scrub_stamp":"2024-04-25T23:54:04.096387+0000","last_deep_scrub":"22'4964","last_deep_scrub_stamp":"2024-04-25T23:54:04.096387+0000","last_clean_scrub_stamp":"2024-04-25T23:54:04.096387+0000","objects_scrubbed":36,"log_size":4964,"log_dups_size":0,"ondisk_log_size":4964,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:57:51.928037+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":149553152,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4881,"num_write_kb":166152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"22'7367","reported_seq":7404,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T23:54:10.868918+0000","last_change":"2024-04-25T23:54:10.869023+0000","last_active":"2024-04-25T23:54:10.868918+0000","last_peered":"2024-04-25T23:54:10.868918+0000","last_clean":"2024-04-25T23:54:10.868918+0000","last_became_active":"2024-04-25T23:49:04.996841+0000","last_became_peered":"2024-04-25T23:49:04.996841+0000","last_unstale":"2024-04-25T23:54:10.868918+0000","last_undegraded":"2024-04-25T23:54:10.868918+0000","last_fullsized":"2024-04-25T23:54:10.868918+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'7367","last_scrub_stamp":"2024-04-25T23:54:10.868810+0000","last_deep_scrub":"22'7367","last_deep_scrub_stamp":"2024-04-25T23:54:10.868810+0000","last_clean_scrub_stamp":"2024-04-25T23:54:10.868810+0000","objects_scrubbed":23,"log_size":7367,"log_dups_size":0,"ondisk_log_size":7367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:49:25.699281+0000","scrub_duration":435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83346448,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":33,"num_read_kb":88,"num_write":7349,"num_write_kb":122645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":3560837368,"num_objects":876,"num_object_clones":0,"num_object_copies":1752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":876,"num_whiteouts":0,"num_read":2566,"num_read_kb":2582,"num_write":155708,"num_write_kb":5620481,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":6782771200,"data_stored":6820324472,"data_compressed":194288578,"data_compressed_allocated":202072064,"data_compressed_original":243892224,"omap_allocated":0,"internal_metadata":0},"log_size":158398,"ondisk_log_size":158398,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":12118,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542211,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":388032,"kb_used_data":298616,"kb_used_omap":1,"kb_used_meta":89406,"kb_avail":93983808,"statfs":{"total":96636764160,"available":96239419392,"internally_reserved":0,"allocated":305782784,"data_stored":306549634,"data_compressed":6553827,"data_compressed_allocated":6823936,"data_compressed_original":7905280,"omap_allocated":1593,"internal_metadata":91552199},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542213,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1212828,"kb_used_data":929044,"kb_used_omap":1,"kb_used_meta":283774,"kb_avail":93159012,"statfs":{"total":96636764160,"available":95394828288,"internally_reserved":0,"allocated":951341056,"data_stored":954367762,"data_compressed":23747576,"data_compressed_allocated":24735744,"data_compressed_original":28565504,"omap_allocated":1595,"internal_metadata":290585029},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542211,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1859000,"kb_used_data":1332336,"kb_used_omap":1,"kb_used_meta":526654,"kb_avail":92512840,"statfs":{"total":96636764160,"available":94733148160,"internally_reserved":0,"allocated":1364312064,"data_stored":1378228858,"data_compressed":53769970,"data_compressed_allocated":55988224,"data_compressed_original":70811648,"omap_allocated":1595,"internal_metadata":539294149},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574915,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1464268,"kb_used_data":1031236,"kb_used_omap":1,"kb_used_meta":433022,"kb_avail":92907572,"statfs":{"total":96636764160,"available":95137353728,"internally_reserved":0,"allocated":1055985664,"data_stored":1063153382,"data_compressed":33533197,"data_compressed_allocated":34906112,"data_compressed_original":43012096,"omap_allocated":1590,"internal_metadata":443414986},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574916,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1422184,"kb_used_data":1062048,"kb_used_omap":1,"kb_used_meta":360126,"kb_avail":92949656,"statfs":{"total":96636764160,"available":95180447744,"internally_reserved":0,"allocated":1087537152,"data_stored":1089959202,"data_compressed":21531874,"data_compressed_allocated":22360064,"data_compressed_original":25559040,"omap_allocated":1593,"internal_metadata":368769479},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574915,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":424760,"kb_used_data":303344,"kb_used_omap":1,"kb_used_meta":121406,"kb_avail":93947080,"statfs":{"total":96636764160,"available":96201809920,"internally_reserved":0,"allocated":310624256,"data_stored":314948974,"data_compressed":11229575,"data_compressed_allocated":11784192,"data_compressed_original":16420864,"omap_allocated":1595,"internal_metadata":124320197},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":861352,"kb_used_data":623392,"kb_used_omap":1,"kb_used_meta":237950,"kb_avail":93510488,"statfs":{"total":96636764160,"available":95754739712,"internally_reserved":0,"allocated":638353408,"data_stored":641447450,"data_compressed":20262986,"data_compressed_allocated":21012480,"data_compressed_original":24612864,"omap_allocated":1595,"internal_metadata":243661253},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574915,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1391768,"kb_used_data":1045520,"kb_used_omap":1,"kb_used_meta":346238,"kb_avail":92980072,"statfs":{"total":96636764160,"available":95211593728,"internally_reserved":0,"allocated":1070612480,"data_stored":1073283690,"data_compressed":23671691,"data_compressed_allocated":24526848,"data_compressed_original":28028928,"omap_allocated":1595,"internal_metadata":354548165},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":6059,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":6059,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1070350336,"data_stored":1072685184,"data_compressed":23665632,"data_compressed_allocated":24494080,"data_compressed_original":27516928,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":638144512,"data_stored":641377872,"data_compressed":20262986,"data_compressed_allocated":21012480,"data_compressed_original":24612864,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":310415360,"data_stored":314879396,"data_compressed":11229575,"data_compressed_allocated":11784192,"data_compressed_original":16420864,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1087328256,"data_stored":1089889624,"data_compressed":21531874,"data_compressed_allocated":22360064,"data_compressed_original":25559040,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1364103168,"data_stored":1378159280,"data_compressed":53769970,"data_compressed_allocated":55988224,"data_compressed_original":70811648,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":951132160,"data_stored":954298184,"data_compressed":23747576,"data_compressed_allocated":24735744,"data_compressed_original":28565504,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":305573888,"data_stored":306480056,"data_compressed":6553827,"data_compressed_allocated":6823936,"data_compressed_original":7905280,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1055723520,"data_stored":1062554876,"data_compressed":33527138,"data_compressed_allocated":34873344,"data_compressed_original":42500096,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T23:54:26.515 DEBUG:teuthology.orchestra.run.smithi057:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-25T23:54:26.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.656+0000 7effc0bd5640 1 Processor -- start 2024-04-25T23:54:26.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.656+0000 7effc0bd5640 1 -- start start 2024-04-25T23:54:26.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.656+0000 7effc0bd5640 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7effbc140650 0x7effbc1444b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:26.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.656+0000 7effc0bd5640 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effbc0b07e0 con 0x7effbc0728a0 2024-04-25T23:54:26.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.656+0000 7effc0bd5640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effbc0737c0 con 0x7effbc0b0400 2024-04-25T23:54:26.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.656+0000 7effc0bd5640 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 -- 0x7effbc144020 con 0x7effbc140650 2024-04-25T23:54:26.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.656+0000 7effbad9d640 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7effbc0728a0 0x7effbc14b3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:60948/0 (socket says 172.21.15.57:60948) 2024-04-25T23:54:26.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.656+0000 7effbad9d640 1 -- 172.21.15.57:0/4052912562 learned_addr learned my addr 172.21.15.57:0/4052912562 (peer_addr_for_me v1:172.21.15.57:0/0) 2024-04-25T23:54:26.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.656+0000 7effb9d9b640 1 --2- 172.21.15.57:0/4052912562 >> v2:172.21.15.57:3300/0 conn(0x7effbc140650 0x7effbc1444b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:26.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.656+0000 7effb959a640 1 -- 172.21.15.57:0/4052912562 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3034599667 0 0) 0x7effb40014a0 con 0x7effbc0728a0 2024-04-25T23:54:26.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.656+0000 7effb959a640 1 -- 172.21.15.57:0/4052912562 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7effa4003530 con 0x7effbc0728a0 2024-04-25T23:54:26.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.656+0000 7effb9d9b640 1 -- 172.21.15.57:0/4052912562 >> v1:172.21.15.80:6789/0 conn(0x7effbc0b0400 legacy=0x7effbc13ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:26.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.656+0000 7effb9d9b640 1 -- 172.21.15.57:0/4052912562 >> v1:172.21.15.57:6789/0 conn(0x7effbc0728a0 legacy=0x7effbc14b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:26.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.656+0000 7effb9d9b640 1 -- 172.21.15.57:0/4052912562 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effbc0734e0 con 0x7effbc140650 2024-04-25T23:54:26.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.660+0000 7effb9d9b640 1 --2- 172.21.15.57:0/4052912562 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7effbc140650 0x7effbc1444b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7effb0009820 tx=0x7effb001c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f71d14fd504e038 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:54:26.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.660+0000 7effb959a640 1 -- 172.21.15.57:0/4052912562 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb00b0070 con 0x7effbc140650 2024-04-25T23:54:26.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.660+0000 7effb959a640 1 -- 172.21.15.57:0/4052912562 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effb00a4e50 con 0x7effbc140650 2024-04-25T23:54:26.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.660+0000 7effb959a640 1 -- 172.21.15.57:0/4052912562 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb00ab820 con 0x7effbc140650 2024-04-25T23:54:26.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.660+0000 7effc0bd5640 1 -- 172.21.15.57:0/4052912562 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7effbc140650 msgr2=0x7effbc1444b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:26.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.660+0000 7effc0bd5640 1 --2- 172.21.15.57:0/4052912562 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7effbc140650 0x7effbc1444b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7effb0009820 tx=0x7effb001c340 comp rx=0 tx=0).stop 2024-04-25T23:54:26.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.660+0000 7effc0bd5640 1 -- 172.21.15.57:0/4052912562 shutdown_connections 2024-04-25T23:54:26.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.660+0000 7effc0bd5640 1 --2- 172.21.15.57:0/4052912562 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7effbc140650 0x7effbc1444b0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:26.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.660+0000 7effc0bd5640 1 -- 172.21.15.57:0/4052912562 >> 172.21.15.57:0/4052912562 conn(0x7effbc0b1290 msgr2=0x7effbc0b16a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:54:26.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.660+0000 7effc0bd5640 1 -- 172.21.15.57:0/4052912562 shutdown_connections 2024-04-25T23:54:26.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.660+0000 7effc0bd5640 1 -- 172.21.15.57:0/4052912562 wait complete. 2024-04-25T23:54:26.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.660+0000 7effc0bd5640 1 Processor -- start 2024-04-25T23:54:26.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.660+0000 7effc0bd5640 1 -- start start 2024-04-25T23:54:26.649 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.664+0000 7effc0bd5640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7effbc0728a0 0x7effbc0b3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:26.649 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.664+0000 7effc0bd5640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7effbc0b0400 0x7effbc1dcf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:26.649 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.664+0000 7effc0bd5640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7effbc140650 0x7effbc135070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:26.649 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.664+0000 7effc0bd5640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7effbc14f880 con 0x7effbc0b0400 2024-04-25T23:54:26.649 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.664+0000 7effc0bd5640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7effbc14f700 con 0x7effbc0728a0 2024-04-25T23:54:26.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.664+0000 7effc0bd5640 1 -- --> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] -- mon_getmap magic: 0 -- 0x7effbc14fa00 con 0x7effbc140650 2024-04-25T23:54:26.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.664+0000 7effb9d9b640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7effbc0b0400 0x7effbc1dcf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:26.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.664+0000 7effb9d9b640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7effbc0b0400 0x7effbc1dcf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53878/0 (socket says 172.21.15.57:53878) 2024-04-25T23:54:26.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.664+0000 7effb9d9b640 1 -- 172.21.15.57:0/1300389846 learned_addr learned my addr 172.21.15.57:0/1300389846 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-04-25T23:54:26.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.664+0000 7effbad9d640 1 --2- >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7effbc140650 0x7effbc135070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:26.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.664+0000 7effba59c640 1 --2- 172.21.15.57:0/1300389846 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7effbc0728a0 0x7effbc0b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:26.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.664+0000 7effb9d9b640 1 -- 172.21.15.57:0/1300389846 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7effbc140650 msgr2=0x7effbc135070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:26.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.664+0000 7effb9d9b640 1 --2- 172.21.15.57:0/1300389846 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7effbc140650 0x7effbc135070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:26.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.664+0000 7effb9d9b640 1 -- 172.21.15.57:0/1300389846 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7effbc0728a0 msgr2=0x7effbc0b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:26.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.664+0000 7effb9d9b640 1 --2- 172.21.15.57:0/1300389846 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7effbc0728a0 0x7effbc0b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:26.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.664+0000 7effb9d9b640 1 -- 172.21.15.57:0/1300389846 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effbc1dd4d0 con 0x7effbc0b0400 2024-04-25T23:54:26.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.664+0000 7effba59c640 1 --2- 172.21.15.57:0/1300389846 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7effbc0728a0 0x7effbc0b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T23:54:26.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.664+0000 7effbad9d640 1 --2- 172.21.15.57:0/1300389846 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7effbc140650 0x7effbc135070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T23:54:26.650 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.664+0000 7effb9d9b640 1 --2- 172.21.15.57:0/1300389846 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7effbc0b0400 0x7effbc1dcf90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7effb0009820 tx=0x7effb001ab80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:54:26.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.664+0000 7effaaffd640 1 -- 172.21.15.57:0/1300389846 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb00bc070 con 0x7effbc0b0400 2024-04-25T23:54:26.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.664+0000 7effaaffd640 1 -- 172.21.15.57:0/1300389846 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effb00b1040 con 0x7effbc0b0400 2024-04-25T23:54:26.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.664+0000 7effc0bd5640 1 -- 172.21.15.57:0/1300389846 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effbc1355b0 con 0x7effbc0b0400 2024-04-25T23:54:26.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.664+0000 7effaaffd640 1 -- 172.21.15.57:0/1300389846 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb00b0050 con 0x7effbc0b0400 2024-04-25T23:54:26.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.664+0000 7effc0bd5640 1 -- 172.21.15.57:0/1300389846 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effbc13d330 con 0x7effbc0b0400 2024-04-25T23:54:26.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.664+0000 7effc0bd5640 1 -- 172.21.15.57:0/1300389846 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effbc0761e0 con 0x7effbc0b0400 2024-04-25T23:54:26.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.664+0000 7effaaffd640 1 -- 172.21.15.57:0/1300389846 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) ==== 87410+0+0 (secure 0 0 0) 0x7effb00bf070 con 0x7effbc0b0400 2024-04-25T23:54:26.652 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.664+0000 7effaaffd640 1 --2- 172.21.15.57:0/1300389846 >> v2:172.21.15.80:6816/2493457677 conn(0x7eff9c0678b0 0x7eff9c069d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T23:54:26.652 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.664+0000 7effaaffd640 1 -- 172.21.15.57:0/1300389846 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 4525+0+0 (secure 0 0 0) 0x7effb011f400 con 0x7effbc0b0400 2024-04-25T23:54:26.652 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.664+0000 7effba59c640 1 --2- 172.21.15.57:0/1300389846 >> v2:172.21.15.80:6816/2493457677 conn(0x7eff9c0678b0 0x7eff9c069d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T23:54:26.652 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.668+0000 7effba59c640 1 --2- 172.21.15.57:0/1300389846 >> v2:172.21.15.80:6816/2493457677 conn(0x7eff9c0678b0 0x7eff9c069d70 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7effbc1dad60 tx=0x7effac009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T23:54:26.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.668+0000 7effaaffd640 1 -- 172.21.15.57:0/1300389846 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7effb00c6020 con 0x7effbc0b0400 2024-04-25T23:54:26.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.832+0000 7effc0bd5640 1 -- 172.21.15.57:0/1300389846 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7effbc0a94e0 con 0x7effbc0b0400 2024-04-25T23:54:26.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.840+0000 7effaaffd640 1 -- 172.21.15.57:0/1300389846 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v5)=0 v5) ==== 123+0+0 (secure 0 0 0) 0x7effb0016050 con 0x7effbc0b0400 2024-04-25T23:54:26.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.840+0000 7effaaffd640 1 -- 172.21.15.57:0/1300389846 <== mon.0 v2:172.21.15.57:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7effb00093e0 con 0x7effbc0b0400 2024-04-25T23:54:26.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.840+0000 7effc0bd5640 1 -- 172.21.15.57:0/1300389846 >> v2:172.21.15.80:6816/2493457677 conn(0x7eff9c0678b0 msgr2=0x7eff9c069d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:26.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.840+0000 7effc0bd5640 1 --2- 172.21.15.57:0/1300389846 >> v2:172.21.15.80:6816/2493457677 conn(0x7eff9c0678b0 0x7eff9c069d70 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7effbc1dad60 tx=0x7effac009040 comp rx=0 tx=0).stop 2024-04-25T23:54:26.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.840+0000 7effc0bd5640 1 -- 172.21.15.57:0/1300389846 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7effbc0b0400 msgr2=0x7effbc1dcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T23:54:26.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.840+0000 7effc0bd5640 1 --2- 172.21.15.57:0/1300389846 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7effbc0b0400 0x7effbc1dcf90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7effb0009820 tx=0x7effb001ab80 comp rx=0 tx=0).stop 2024-04-25T23:54:26.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.840+0000 7effc0bd5640 1 -- 172.21.15.57:0/1300389846 shutdown_connections 2024-04-25T23:54:26.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.840+0000 7effc0bd5640 1 --2- 172.21.15.57:0/1300389846 >> v2:172.21.15.80:6816/2493457677 conn(0x7eff9c0678b0 0x7eff9c069d70 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:26.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.840+0000 7effc0bd5640 1 --2- 172.21.15.57:0/1300389846 >> [v2:172.21.15.57:3301/0,v1:172.21.15.57:6790/0] conn(0x7effbc140650 0x7effbc135070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:26.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.840+0000 7effc0bd5640 1 --2- 172.21.15.57:0/1300389846 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7effbc0b0400 0x7effbc1dcf90 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:26.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.840+0000 7effc0bd5640 1 --2- 172.21.15.57:0/1300389846 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7effbc0728a0 0x7effbc0b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T23:54:26.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.840+0000 7effc0bd5640 1 -- 172.21.15.57:0/1300389846 >> 172.21.15.57:0/1300389846 conn(0x7effbc0b1290 msgr2=0x7effbc133810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T23:54:26.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.844+0000 7effc0bd5640 1 -- 172.21.15.57:0/1300389846 shutdown_connections 2024-04-25T23:54:26.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-04-25T23:54:26.844+0000 7effc0bd5640 1 -- 172.21.15.57:0/1300389846 wait complete. 2024-04-25T23:54:26.844 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-25T23:54:26.844 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-25T23:54:26.844 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-25T23:54:26.844 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T23:54:26.934 INFO:tasks.ceph.osd.0:Stopped 2024-04-25T23:54:26.934 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-25T23:54:26.935 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T23:54:27.027 INFO:tasks.ceph.osd.1:Stopped 2024-04-25T23:54:27.028 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-25T23:54:27.028 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T23:54:27.118 INFO:tasks.ceph.osd.2:Stopped 2024-04-25T23:54:27.119 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-25T23:54:27.119 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T23:54:28.126 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-04-25T23:54:33.220 INFO:tasks.ceph.osd.3:Stopped 2024-04-25T23:54:33.220 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-25T23:54:33.221 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T23:54:34.035 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-04-25T23:54:34.035 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-04-25T23:54:39.322 INFO:tasks.ceph.osd.4:Stopped 2024-04-25T23:54:39.323 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-25T23:54:39.323 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T23:54:39.387 INFO:tasks.ceph.osd.5:Stopped 2024-04-25T23:54:39.387 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-25T23:54:39.387 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T23:54:39.476 INFO:tasks.ceph.osd.6:Stopped 2024-04-25T23:54:39.476 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-25T23:54:39.476 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T23:54:39.841 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2024-04-25T23:54:39.842 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-04-25T23:54:45.346 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2024-04-25T23:54:45.346 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 has been restored 2024-04-25T23:54:45.578 INFO:tasks.ceph.osd.7:Stopped 2024-04-25T23:54:45.578 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-25T23:54:45.579 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-25T23:54:45.579 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T23:54:45.608 INFO:tasks.ceph.mgr.y:Stopped 2024-04-25T23:54:45.608 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-25T23:54:45.608 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T23:54:45.639 INFO:tasks.ceph.mgr.x:Stopped 2024-04-25T23:54:45.639 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-25T23:54:45.639 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-25T23:54:45.639 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T23:54:45.665 INFO:tasks.ceph.mon.a:Stopped 2024-04-25T23:54:45.665 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-25T23:54:45.666 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T23:54:45.725 INFO:tasks.ceph.mon.c:Stopped 2024-04-25T23:54:45.749 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-25T23:54:45.750 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T23:54:45.768 INFO:tasks.ceph.mon.b:Stopped 2024-04-25T23:54:45.768 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-25T23:54:45.769 DEBUG:teuthology.orchestra.run.smithi057:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | head -n 1 2024-04-25T23:54:45.789 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi057.front.sepia.ceph.com 2024-04-25T23:54:45.789 DEBUG:teuthology.orchestra.run.smithi057:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-25T23:54:46.203 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi057.front.sepia.ceph.com 2024-04-25T23:54:46.203 DEBUG:teuthology.orchestra.run.smithi057:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-25T23:54:46.380 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi057.front.sepia.ceph.com 2024-04-25T23:54:46.380 DEBUG:teuthology.orchestra.run.smithi057:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-25T23:54:46.515 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi057.front.sepia.ceph.com 2024-04-25T23:54:46.515 DEBUG:teuthology.orchestra.run.smithi057:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-25T23:54:46.657 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi080.front.sepia.ceph.com 2024-04-25T23:54:46.657 DEBUG:teuthology.orchestra.run.smithi080:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-25T23:54:46.913 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi080.front.sepia.ceph.com 2024-04-25T23:54:46.913 DEBUG:teuthology.orchestra.run.smithi080:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-25T23:54:47.115 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi080.front.sepia.ceph.com 2024-04-25T23:54:47.115 DEBUG:teuthology.orchestra.run.smithi080:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-25T23:54:47.245 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi080.front.sepia.ceph.com 2024-04-25T23:54:47.246 DEBUG:teuthology.orchestra.run.smithi080:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-25T23:54:47.418 INFO:tasks.ceph:Archiving mon data... 2024-04-25T23:54:47.419 DEBUG:teuthology.misc:Transferring archived files from smithi057:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-04-25_21:00:03-rados-squid-distro-default-smithi/7673825/data/mon.a.tgz 2024-04-25T23:54:47.420 DEBUG:teuthology.orchestra.run.smithi057:> mktemp 2024-04-25T23:54:47.426 INFO:teuthology.orchestra.run.smithi057.stdout:/tmp/tmp.nMyjlTI3M2 2024-04-25T23:54:47.426 DEBUG:teuthology.orchestra.run.smithi057:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.nMyjlTI3M2 2024-04-25T23:54:47.542 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod 0666 /tmp/tmp.nMyjlTI3M2 2024-04-25T23:54:47.602 DEBUG:teuthology.orchestra.remote:smithi057:/tmp/tmp.nMyjlTI3M2 is 309KB 2024-04-25T23:54:47.660 DEBUG:teuthology.orchestra.run.smithi057:> rm -fr /tmp/tmp.nMyjlTI3M2 2024-04-25T23:54:47.666 DEBUG:teuthology.misc:Transferring archived files from smithi057:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-04-25_21:00:03-rados-squid-distro-default-smithi/7673825/data/mon.c.tgz 2024-04-25T23:54:47.666 DEBUG:teuthology.orchestra.run.smithi057:> mktemp 2024-04-25T23:54:47.715 INFO:teuthology.orchestra.run.smithi057.stdout:/tmp/tmp.ZlvRWB3DS4 2024-04-25T23:54:47.715 DEBUG:teuthology.orchestra.run.smithi057:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.ZlvRWB3DS4 2024-04-25T23:54:47.830 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod 0666 /tmp/tmp.ZlvRWB3DS4 2024-04-25T23:54:47.895 DEBUG:teuthology.orchestra.remote:smithi057:/tmp/tmp.ZlvRWB3DS4 is 311KB 2024-04-25T23:54:47.956 DEBUG:teuthology.orchestra.run.smithi057:> rm -fr /tmp/tmp.ZlvRWB3DS4 2024-04-25T23:54:47.962 DEBUG:teuthology.misc:Transferring archived files from smithi080:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-04-25_21:00:03-rados-squid-distro-default-smithi/7673825/data/mon.b.tgz 2024-04-25T23:54:47.962 DEBUG:teuthology.orchestra.run.smithi080:> mktemp 2024-04-25T23:54:47.967 INFO:teuthology.orchestra.run.smithi080.stdout:/tmp/tmp.Pa4YIAjrya 2024-04-25T23:54:47.968 DEBUG:teuthology.orchestra.run.smithi080:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.Pa4YIAjrya 2024-04-25T23:54:48.081 DEBUG:teuthology.orchestra.run.smithi080:> sudo chmod 0666 /tmp/tmp.Pa4YIAjrya 2024-04-25T23:54:48.141 DEBUG:teuthology.orchestra.remote:smithi080:/tmp/tmp.Pa4YIAjrya is 311KB 2024-04-25T23:54:48.199 DEBUG:teuthology.orchestra.run.smithi080:> rm -fr /tmp/tmp.Pa4YIAjrya 2024-04-25T23:54:48.204 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-25T23:54:48.204 DEBUG:teuthology.orchestra.run.smithi057:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T23:54:48.206 DEBUG:teuthology.orchestra.run.smithi080:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T23:54:48.267 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T23:54:48.301 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T23:54:48.301 INFO:tasks.ceph:Archiving crash dumps... 2024-04-25T23:54:48.302 DEBUG:teuthology.misc:Transferring archived files from smithi057:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-04-25_21:00:03-rados-squid-distro-default-smithi/7673825/remote/smithi057/crash 2024-04-25T23:54:48.303 DEBUG:teuthology.orchestra.run.smithi057:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T23:54:48.323 DEBUG:teuthology.misc:Transferring archived files from smithi080:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-04-25_21:00:03-rados-squid-distro-default-smithi/7673825/remote/smithi080/crash 2024-04-25T23:54:48.324 DEBUG:teuthology.orchestra.run.smithi080:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T23:54:48.357 INFO:tasks.ceph:Compressing logs... 2024-04-25T23:54:48.357 DEBUG:teuthology.orchestra.run.smithi057:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T23:54:48.372 DEBUG:teuthology.orchestra.run.smithi080:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T23:54:48.393 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29286.log 2024-04-25T23:54:48.393 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.libcephsqlite.29756.log 2024-04-25T23:54:48.394 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.29286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30143.log 2024-04-25T23:54:48.394 INFO:teuthology.orchestra.run.smithi057.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.29286.log.gz 2024-04-25T23:54:48.395 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.libcephsqlite.29756.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-25T23:54:48.395 INFO:teuthology.orchestra.run.smithi057.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.libcephsqlite.29756.log.gz 2024-04-25T23:54:48.395 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.30143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30377.log 2024-04-25T23:54:48.396 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- 87.2% /var/log/ceph/ceph-client.admin.29026.log 2024-04-25T23:54:48.396 INFO:teuthology.orchestra.run.smithi057.stderr: -- replaced with /var/log/ceph/ceph-client.admin.30143.log.gz 2024-04-25T23:54:48.396 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.30377.log: /var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30247.log 2024-04-25T23:54:48.396 INFO:teuthology.orchestra.run.smithi057.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.30377.log.gz 2024-04-25T23:54:48.397 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.29026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30039.log 2024-04-25T23:54:48.397 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.30247.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.29026.log.gz 2024-04-25T23:54:48.397 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28824.log 2024-04-25T23:54:48.397 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.30039.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.30247.log.gz 2024-04-25T23:54:48.397 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30583.log 2024-04-25T23:54:48.399 INFO:teuthology.orchestra.run.smithi057.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.30039.log.gz 2024-04-25T23:54:48.399 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.28824.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24983.log 2024-04-25T23:54:48.399 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.30583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25240.log 2024-04-25T23:54:48.399 INFO:teuthology.orchestra.run.smithi057.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.28824.log.gz 2024-04-25T23:54:48.399 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.24983.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.30583.log.gz 2024-04-25T23:54:48.400 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-25T23:54:48.400 INFO:teuthology.orchestra.run.smithi057.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.admin.24983.log.gz 2024-04-25T23:54:48.400 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.25240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28564.log 2024-04-25T23:54:48.400 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-osd.0.log: gzip 87.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.25240.log.gz -- 2024-04-25T23:54:48.400 INFO:teuthology.orchestra.run.smithi057.stderr: /var/log/ceph/ceph-client.admin.30273.log 2024-04-25T23:54:48.401 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.28564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30481.log 2024-04-25T23:54:48.401 INFO:teuthology.orchestra.run.smithi057.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.28564.log.gz 2024-04-25T23:54:48.401 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.30273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28626.log 2024-04-25T23:54:48.401 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.30481.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.30273.log.gz 2024-04-25T23:54:48.402 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28822.log 2024-04-25T23:54:48.402 INFO:teuthology.orchestra.run.smithi057.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.30481.log.gz 2024-04-25T23:54:48.402 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.28626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28625.log 2024-04-25T23:54:48.402 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.28822.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.28626.log.gz 2024-04-25T23:54:48.403 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-25T23:54:48.403 INFO:teuthology.orchestra.run.smithi057.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.28822.log.gz 2024-04-25T23:54:48.403 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30507.log 2024-04-25T23:54:48.403 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.28625.log: 93.5% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-25T23:54:48.403 INFO:teuthology.orchestra.run.smithi057.stderr:gzip/var/log/ceph/ceph-osd.2.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.28470.log 2024-04-25T23:54:48.404 INFO:teuthology.orchestra.run.smithi057.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.28625.log.gz 2024-04-25T23:54:48.404 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.30507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28609.log 2024-04-25T23:54:48.404 INFO:teuthology.orchestra.run.smithi057.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.30507.log.gz 2024-04-25T23:54:48.404 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.28470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29260.log 2024-04-25T23:54:48.405 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.28609.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.28470.log.gz 2024-04-25T23:54:48.405 INFO:teuthology.orchestra.run.smithi057.stderr: 88.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.28609.log.gz 2024-04-25T23:54:48.405 INFO:teuthology.orchestra.run.smithi057.stderr: -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-25T23:54:48.405 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30455.log 2024-04-25T23:54:48.405 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.29260.log: /var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29054.log 2024-04-25T23:54:48.405 INFO:teuthology.orchestra.run.smithi057.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.29260.log.gz 2024-04-25T23:54:48.405 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.30455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28825.log 2024-04-25T23:54:48.407 INFO:teuthology.orchestra.run.smithi057.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.30455.log.gz 2024-04-25T23:54:48.407 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.29054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29034.log 2024-04-25T23:54:48.407 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28611.log 2024-04-25T23:54:48.408 INFO:teuthology.orchestra.run.smithi057.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.29054.log.gz 2024-04-25T23:54:48.408 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.29034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29046.log 2024-04-25T23:54:48.408 INFO:teuthology.orchestra.run.smithi057.stderr: 87.9%/var/log/ceph/ceph-client.admin.28611.log: -- replaced with /var/log/ceph/ceph-client.admin.29034.log.gz 2024-04-25T23:54:48.408 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.28825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30117.log 2024-04-25T23:54:48.408 INFO:teuthology.orchestra.run.smithi057.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.28611.log.gz 2024-04-25T23:54:48.408 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.29046.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.28825.log.gz 2024-04-25T23:54:48.408 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28620.log 2024-04-25T23:54:48.412 INFO:teuthology.orchestra.run.smithi057.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.29046.log.gz 2024-04-25T23:54:48.412 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.30117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28623.log 2024-04-25T23:54:48.412 INFO:teuthology.orchestra.run.smithi057.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.30117.log.gz 2024-04-25T23:54:48.412 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.28620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30169.log 2024-04-25T23:54:48.412 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30403.log 2024-04-25T23:54:48.412 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.28623.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.28620.log.gz 2024-04-25T23:54:48.412 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.30169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30559.log 2024-04-25T23:54:48.412 INFO:teuthology.orchestra.run.smithi057.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.28623.log.gz 2024-04-25T23:54:48.412 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.30403.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.30169.log.gz 2024-04-25T23:54:48.412 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-25T23:54:48.413 INFO:teuthology.orchestra.run.smithi057.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.30403.log.gz 2024-04-25T23:54:48.413 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.30559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30091.log 2024-04-25T23:54:48.413 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.20207.log 2024-04-25T23:54:48.413 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-osd.3.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.30559.log.gz 2024-04-25T23:54:48.413 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.30091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28814.log 2024-04-25T23:54:48.413 INFO:teuthology.orchestra.run.smithi057.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.30091.log.gz 2024-04-25T23:54:48.413 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph.tmp-client.admin.20207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.libcephsqlite.29852.log 0.0% 2024-04-25T23:54:48.413 INFO:teuthology.orchestra.run.smithi057.stderr: -- replaced with /var/log/ceph/ceph.tmp-client.admin.20207.log.gz 2024-04-25T23:54:48.413 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30351.log 2024-04-25T23:54:48.416 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.28814.log: /var/log/ceph/ceph-client.libcephsqlite.29852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28830.log 2024-04-25T23:54:48.416 INFO:teuthology.orchestra.run.smithi057.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.28814.log.gz/var/log/ceph/ceph-client.admin.30351.log: 2024-04-25T23:54:48.416 INFO:teuthology.orchestra.run.smithi057.stderr: 90.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.libcephsqlite.29984.log 2024-04-25T23:54:48.416 INFO:teuthology.orchestra.run.smithi057.stderr: -- replaced with /var/log/ceph/ceph-client.libcephsqlite.29852.log.gz 2024-04-25T23:54:48.417 INFO:teuthology.orchestra.run.smithi057.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.30351.log.gz 2024-04-25T23:54:48.417 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.28830.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-25T23:54:48.417 INFO:teuthology.orchestra.run.smithi057.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.28830.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25137.log 2024-04-25T23:54:48.417 INFO:teuthology.orchestra.run.smithi057.stderr: 2024-04-25T23:54:48.417 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.libcephsqlite.29984.log: /var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28622.log 2024-04-25T23:54:48.417 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30195.log 2024-04-25T23:54:48.417 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.25137.log: /var/log/ceph/ceph-client.admin.28622.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.25137.log.gz 2024-04-25T23:54:48.417 INFO:teuthology.orchestra.run.smithi057.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.28622.log.gz 2024-04-25T23:54:48.417 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28826.log 2024-04-25T23:54:48.417 INFO:teuthology.orchestra.run.smithi057.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.libcephsqlite.29984.log.gz 2024-04-25T23:54:48.418 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.30195.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.30195.log.gz 2024-04-25T23:54:48.419 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28840.log 2024-04-25T23:54:48.419 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.28826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.libcephsqlite.29920.log 2024-04-25T23:54:48.419 INFO:teuthology.orchestra.run.smithi057.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.28826.log.gz 2024-04-25T23:54:48.419 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.28840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28616.log 2024-04-25T23:54:48.419 INFO:teuthology.orchestra.run.smithi057.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.28840.log.gz 2024-04-25T23:54:48.419 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.libcephsqlite.29920.log: 89.6% -- replaced with /var/log/ceph/ceph-client.libcephsqlite.29920.log.gz 2024-04-25T23:54:48.420 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30299.log 2024-04-25T23:54:48.421 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.28616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.libcephsqlite.29901.log 2024-04-25T23:54:48.421 INFO:teuthology.orchestra.run.smithi057.stderr: 87.8%/var/log/ceph/ceph-client.admin.30299.log: -- replaced with /var/log/ceph/ceph-client.admin.28616.log.gz 2024-04-25T23:54:48.421 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T23:54:48.421 INFO:teuthology.orchestra.run.smithi057.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.30299.log.gz 2024-04-25T23:54:48.421 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29033.log 2024-04-25T23:54:48.423 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.libcephsqlite.29901.log: 92.3% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T23:54:48.423 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29044.log 2024-04-25T23:54:48.424 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.29033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30065.log 2024-04-25T23:54:48.425 INFO:teuthology.orchestra.run.smithi057.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.29033.log.gz 2024-04-25T23:54:48.425 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.29044.log: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30325.log 2024-04-25T23:54:48.425 INFO:teuthology.orchestra.run.smithi057.stderr: -- replaced with /var/log/ceph/ceph-client.admin.29044.log.gz 2024-04-25T23:54:48.426 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.30065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28823.log 2024-04-25T23:54:48.426 INFO:teuthology.orchestra.run.smithi057.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.30065.log.gz 2024-04-25T23:54:48.427 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.30325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29045.log 2024-04-25T23:54:48.427 INFO:teuthology.orchestra.run.smithi057.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.30325.log.gz 2024-04-25T23:54:48.427 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.28823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.libcephsqlite.29875.log 2024-04-25T23:54:48.428 INFO:teuthology.orchestra.run.smithi057.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.28823.log.gz 2024-04-25T23:54:48.428 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.29045.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-25T23:54:48.428 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.libcephsqlite.29875.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.29045.log.gz 2024-04-25T23:54:48.429 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.libcephsqlite.29800.log 2024-04-25T23:54:48.429 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25170.log 2024-04-25T23:54:48.430 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25242.log 2024-04-25T23:54:48.430 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.25170.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-25T23:54:48.430 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.25242.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.25170.log.gz 2024-04-25T23:54:48.431 INFO:teuthology.orchestra.run.smithi080.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.25242.log.gz 2024-04-25T23:54:48.431 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-25T23:54:48.431 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25273.log 2024-04-25T23:54:48.432 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-osd.7.log: /var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-25T23:54:48.432 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.25273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25218.log 2024-04-25T23:54:48.432 INFO:teuthology.orchestra.run.smithi080.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.25273.log.gz 2024-04-25T23:54:48.433 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25146.log 2024-04-25T23:54:48.433 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.25218.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-25T23:54:48.433 INFO:teuthology.orchestra.run.smithi080.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.25218.log.gz 2024-04-25T23:54:48.434 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.25146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25194.log 2024-04-25T23:54:48.434 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-mgr.x.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.25146.log.gz 2024-04-25T23:54:48.434 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T23:54:48.434 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.25194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25098.log 2024-04-25T23:54:48.436 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph.log: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T23:54:48.436 INFO:teuthology.orchestra.run.smithi080.stderr: -- replaced with /var/log/ceph/ceph-client.admin.25194.log.gz 2024-04-25T23:54:48.436 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.25098.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-25T23:54:48.436 INFO:teuthology.orchestra.run.smithi080.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.25098.log.gz 2024-04-25T23:54:48.436 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-25T23:54:48.437 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25122.log 2024-04-25T23:54:48.437 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-mon.b.log: 86.1% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T23:54:48.437 INFO:teuthology.orchestra.run.smithi080.stderr: 88.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T23:54:48.438 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-osd.4.log: /var/log/ceph/ceph-client.admin.25122.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.25122.log.gz 2024-04-25T23:54:48.441 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-mon.c.log: 96.0% -- replaced with /var/log/ceph/ceph-client.libcephsqlite.29901.log.gz 2024-04-25T23:54:48.442 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30221.log 2024-04-25T23:54:48.454 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.libcephsqlite.29800.log: 92.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29310.log 2024-04-25T23:54:48.455 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.30221.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.30221.log.gz 2024-04-25T23:54:48.455 INFO:teuthology.orchestra.run.smithi057.stderr: -- replaced with /var/log/ceph/ceph-client.libcephsqlite.29800.log.gz 2024-04-25T23:54:48.456 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28496.log 2024-04-25T23:54:48.457 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.29310.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.29310.log.gz 2024-04-25T23:54:48.457 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T23:54:48.458 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.28496.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.28496.log.gz 2024-04-25T23:54:48.459 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.libcephsqlite.29713.log 2024-04-25T23:54:48.461 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph.audit.log: 93.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T23:54:48.461 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29055.log 2024-04-25T23:54:48.462 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.libcephsqlite.29713.log: 89.7% -- replaced with /var/log/ceph/ceph-client.libcephsqlite.29713.log.gz 2024-04-25T23:54:48.463 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.libcephsqlite.29735.log 2024-04-25T23:54:48.464 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.29055.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.29055.log.gz 2024-04-25T23:54:48.464 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.libcephsqlite.29832.log 2024-04-25T23:54:48.467 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.libcephsqlite.29735.log: 91.6% -- replaced with /var/log/ceph/ceph-client.libcephsqlite.29735.log.gz 2024-04-25T23:54:48.468 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30429.log 2024-04-25T23:54:48.482 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.libcephsqlite.29832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.libcephsqlite.29777.log 2024-04-25T23:54:48.484 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.30429.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.30429.log.gz 2024-04-25T23:54:48.484 INFO:teuthology.orchestra.run.smithi057.stderr: 95.0% -- replaced with /var/log/ceph/ceph-client.libcephsqlite.29832.log.gz 2024-04-25T23:54:48.485 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29337.log 2024-04-25T23:54:48.486 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.libcephsqlite.29777.log: 89.8% -- replaced with /var/log/ceph/ceph-client.libcephsqlite.29777.log.gz 2024-04-25T23:54:48.486 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28418.log 2024-04-25T23:54:48.487 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.29337.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.29337.log.gz 2024-04-25T23:54:48.488 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.libcephsqlite.29943.log 2024-04-25T23:54:48.489 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.28418.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.28418.log.gz 2024-04-25T23:54:48.489 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28522.log 2024-04-25T23:54:48.502 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.libcephsqlite.29943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.libcephsqlite.29361.log 2024-04-25T23:54:48.504 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.28522.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.28522.log.gz 2024-04-25T23:54:48.505 INFO:teuthology.orchestra.run.smithi080.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-25T23:54:48.518 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30533.log 2024-04-25T23:54:48.530 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.libcephsqlite.29361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29234.log 2024-04-25T23:54:48.531 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.30533.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.30533.log.gz 2024-04-25T23:54:48.542 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30013.log 2024-04-25T23:54:48.543 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.29234.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.29234.log.gz 2024-04-25T23:54:48.550 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28444.log 2024-04-25T23:54:48.551 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.30013.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.30013.log.gz 2024-04-25T23:54:48.559 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.28444.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.28444.log.gz 2024-04-25T23:54:48.577 INFO:teuthology.orchestra.run.smithi057.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-25T23:54:48.583 INFO:teuthology.orchestra.run.smithi080.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-25T23:54:48.709 INFO:teuthology.orchestra.run.smithi057.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-25T23:54:48.913 INFO:teuthology.orchestra.run.smithi057.stderr: 95.7% -- replaced with /var/log/ceph/ceph-client.libcephsqlite.29875.log.gz 2024-04-25T23:54:49.365 INFO:teuthology.orchestra.run.smithi057.stderr: 95.7% -- replaced with /var/log/ceph/ceph-client.libcephsqlite.29943.log.gz 2024-04-25T23:54:50.347 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2024-04-25T23:54:52.301 INFO:teuthology.orchestra.run.smithi057.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-25T23:54:52.588 INFO:teuthology.orchestra.run.smithi057.stderr: 95.5% -- replaced with /var/log/ceph/ceph-client.libcephsqlite.29361.log.gz 2024-04-25T23:54:52.885 INFO:teuthology.orchestra.run.smithi080.stderr: 92.9% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-25T23:54:59.147 INFO:teuthology.orchestra.run.smithi057.stderr: 92.4% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-25T23:55:01.865 INFO:teuthology.orchestra.run.smithi080.stderr: 92.5% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-25T23:55:06.551 INFO:teuthology.orchestra.run.smithi057.stderr: 92.2% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-25T23:55:06.672 INFO:teuthology.orchestra.run.smithi080.stderr: 92.4% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-25T23:55:06.806 INFO:teuthology.orchestra.run.smithi057.stderr: 92.5% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-25T23:55:06.808 INFO:teuthology.orchestra.run.smithi057.stderr: 2024-04-25T23:55:06.809 INFO:teuthology.orchestra.run.smithi057.stderr:real 0m18.434s 2024-04-25T23:55:06.809 INFO:teuthology.orchestra.run.smithi057.stderr:user 0m55.360s 2024-04-25T23:55:06.809 INFO:teuthology.orchestra.run.smithi057.stderr:sys 0m1.803s 2024-04-25T23:55:06.898 INFO:teuthology.orchestra.run.smithi080.stderr: 92.4% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-25T23:55:06.899 INFO:teuthology.orchestra.run.smithi080.stderr: 2024-04-25T23:55:06.899 INFO:teuthology.orchestra.run.smithi080.stderr:real 0m18.490s 2024-04-25T23:55:06.899 INFO:teuthology.orchestra.run.smithi080.stderr:user 0m53.130s 2024-04-25T23:55:06.899 INFO:teuthology.orchestra.run.smithi080.stderr:sys 0m1.711s 2024-04-25T23:55:06.900 INFO:tasks.ceph:Archiving logs... 2024-04-25T23:55:06.900 DEBUG:teuthology.misc:Transferring archived files from smithi057:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-25_21:00:03-rados-squid-distro-default-smithi/7673825/remote/smithi057/log 2024-04-25T23:55:06.902 DEBUG:teuthology.orchestra.run.smithi057:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T23:55:13.912 DEBUG:teuthology.misc:Transferring archived files from smithi080:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-25_21:00:03-rados-squid-distro-default-smithi/7673825/remote/smithi080/log 2024-04-25T23:55:13.913 DEBUG:teuthology.orchestra.run.smithi080:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T23:55:21.775 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T23:55:21.785 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T23:55:21.785 DEBUG:teuthology.orchestra.run.smithi057:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T23:55:21.788 DEBUG:teuthology.orchestra.run.smithi080:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T23:55:21.858 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T23:55:21.867 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T23:55:21.867 DEBUG:teuthology.orchestra.run.smithi057:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T23:55:21.869 DEBUG:teuthology.orchestra.run.smithi080:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T23:55:21.881 INFO:teuthology.orchestra.run.smithi057.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T23:55:21.881 INFO:teuthology.orchestra.run.smithi057.stdout:============================================================================== 2024-04-25T23:55:21.881 INFO:teuthology.orchestra.run.smithi057.stdout:+hv01.front.sepi 67.205.162.81 3 u 27 64 377 0.081 -2.400 1.049 2024-04-25T23:55:21.881 INFO:teuthology.orchestra.run.smithi057.stdout:+hv02.front.sepi 63.231.80.2 3 u 39 64 377 0.093 -0.277 1.144 2024-04-25T23:55:21.881 INFO:teuthology.orchestra.run.smithi057.stdout:*hv03.front.sepi 74.6.168.72 3 u 30 64 377 0.089 -1.021 1.028 2024-04-25T23:55:21.882 INFO:teuthology.orchestra.run.smithi057.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T23:55:21.885 INFO:teuthology.orchestra.run.smithi080.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T23:55:21.886 INFO:teuthology.orchestra.run.smithi080.stdout:============================================================================== 2024-04-25T23:55:21.886 INFO:teuthology.orchestra.run.smithi080.stdout:+hv01.front.sepi 67.205.162.81 3 u 35 64 377 0.066 -2.137 0.868 2024-04-25T23:55:21.886 INFO:teuthology.orchestra.run.smithi080.stdout:*hv02.front.sepi 63.231.80.2 3 u 34 64 377 0.083 -0.836 0.867 2024-04-25T23:55:21.886 INFO:teuthology.orchestra.run.smithi080.stdout:+hv03.front.sepi 74.6.168.72 3 u 31 64 377 0.076 -0.491 1.020 2024-04-25T23:55:21.886 INFO:teuthology.orchestra.run.smithi080.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T23:55:21.887 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T23:55:21.896 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T23:55:21.896 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T23:55:21.906 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T23:55:21.921 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T23:55:21.934 INFO:teuthology.task.internal:Duration was 1021.025838 seconds 2024-04-25T23:55:21.934 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T23:55:21.947 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T23:55:21.948 DEBUG:teuthology.orchestra.run.smithi057:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T23:55:21.951 DEBUG:teuthology.orchestra.run.smithi080:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T23:55:21.988 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T23:55:21.988 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi057.front.sepia.ceph.com 2024-04-25T23:55:21.988 DEBUG:teuthology.orchestra.run.smithi057:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T23:55:22.044 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi080.front.sepia.ceph.com 2024-04-25T23:55:22.044 DEBUG:teuthology.orchestra.run.smithi080:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T23:55:22.054 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T23:55:22.055 DEBUG:teuthology.orchestra.run.smithi057:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T23:55:22.088 DEBUG:teuthology.orchestra.run.smithi080:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T23:55:22.121 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T23:55:22.121 DEBUG:teuthology.orchestra.run.smithi057:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T23:55:22.156 DEBUG:teuthology.orchestra.run.smithi080:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T23:55:22.227 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T23:55:22.238 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T23:55:22.238 DEBUG:teuthology.orchestra.run.smithi057:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T23:55:22.251 DEBUG:teuthology.orchestra.run.smithi080:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T23:55:22.262 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T23:55:22.272 DEBUG:teuthology.orchestra.run.smithi057:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T23:55:22.296 DEBUG:teuthology.orchestra.run.smithi080:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T23:55:22.306 INFO:teuthology.orchestra.run.smithi057.stdout:kernel.core_pattern = core 2024-04-25T23:55:22.315 INFO:teuthology.orchestra.run.smithi080.stdout:kernel.core_pattern = core 2024-04-25T23:55:22.344 DEBUG:teuthology.orchestra.run.smithi057:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T23:55:22.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T23:55:22.366 DEBUG:teuthology.orchestra.run.smithi080:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T23:55:22.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T23:55:22.392 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T23:55:22.403 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T23:55:22.404 DEBUG:teuthology.misc:Transferring archived files from smithi057:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-25_21:00:03-rados-squid-distro-default-smithi/7673825/remote/smithi057 2024-04-25T23:55:22.404 DEBUG:teuthology.orchestra.run.smithi057:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T23:55:22.433 DEBUG:teuthology.misc:Transferring archived files from smithi080:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-25_21:00:03-rados-squid-distro-default-smithi/7673825/remote/smithi080 2024-04-25T23:55:22.433 DEBUG:teuthology.orchestra.run.smithi080:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T23:55:22.457 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T23:55:22.457 DEBUG:teuthology.orchestra.run.smithi057:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T23:55:22.472 DEBUG:teuthology.orchestra.run.smithi080:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T23:55:22.501 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T23:55:22.512 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T23:55:22.513 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T23:55:22.523 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T23:55:22.523 DEBUG:teuthology.orchestra.run.smithi057:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T23:55:22.526 DEBUG:teuthology.orchestra.run.smithi080:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T23:55:22.529 INFO:teuthology.orchestra.run.smithi057.stdout: 658614 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 23:55 /home/ubuntu/cephtest 2024-04-25T23:55:22.549 INFO:teuthology.orchestra.run.smithi080.stdout: 658670 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 23:55 /home/ubuntu/cephtest 2024-04-25T23:55:22.550 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T23:55:22.561 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T23:55:22.585 INFO:teuthology.run:Summary data: description: rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} tasks/libcephsqlite} duration: 1021.025838136673 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-04-25T23:55:22.585 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T23:55:22.686 INFO:teuthology.run:pass