2024-09-15T03:25:19.369 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-15T03:25:19.390 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-15T03:25:19.477 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905284 branch: wip-bharath9-testing-2024-09-13-1444-quincy description: rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} tasks/libcephsqlite} email: skanta@redhat.com first_in_suite: false job_id: '7905284' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 - volumes: count: 4 size: 10 os_type: ubuntu os_version: '20.04' overrides: admin_socket: branch: wip-bharath9-testing-2024-09-13-1444-quincy ceph: conf: client: debug cephsqlite: 20 debug client: 20 debug ms: 1 global: mon client directed command retry: 5 mon election default strategy: 1 ms inject socket failures: 5000 ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 mon scrub interval: 300 paxos service trim min: 10 osd: bluefs allocator: hybrid bluestore allocator: hybrid bluestore block size: 96636764160 bluestore fsck on mount: true bluestore zero block detection: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd mclock override recovery settings: true osd mclock profile: high_recovery_ops osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - POOL_APP_NOT_ENABLED - do not have an application enabled sha1: d75f23820806da622f561ee5ee7551015285993f ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: d75f23820806da622f561ee5ee7551015285993f thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-bharath9-testing-2024-09-13-1444-quincy sha1: d75f23820806da622f561ee5ee7551015285993f owner: scheduled_skanta@teuthology priority: 99 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 2126 sha1: d75f23820806da622f561ee5ee7551015285993f sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-bharath9-testing-2024-09-13-1444-quincy suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_d75f23820806da622f561ee5ee7551015285993f/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: d75f23820806da622f561ee5ee7551015285993f targets: smithi110.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= smithi177.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= tasks: - install: extra_system_packages: deb: - sqlite3 rpm: - sqlite-devel - ceph: null - exec: client.0: - ceph osd pool create cephsqlite - ceph auth get-or-create client.libcephsqlite mon 'profile simple-rados-client-with-blocklist' osd 'allow rwx pool=cephsqlite' >> /etc/ceph/ceph.keyring - exec: client.0: - ceph_test_libcephsqlite --id libcephsqlite --no-log-to-stderr - workunit: clients: client.0: - rados/test_libcephsqlite.sh cephsqlite env: CEPH_ARGS: --id libcephsqlite --no-log-to-stderr teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-15_00:54:39 tube: smithi user: skanta verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-15T03:25:19.477 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_d75f23820806da622f561ee5ee7551015285993f/qa; will attempt to use it 2024-09-15T03:25:19.478 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_d75f23820806da622f561ee5ee7551015285993f/qa/tasks 2024-09-15T03:25:19.478 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-15T03:25:19.480 INFO:teuthology.task.internal:Checking packages... 2024-09-15T03:25:19.514 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'd75f23820806da622f561ee5ee7551015285993f' 2024-09-15T03:25:19.514 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-15T03:25:19.515 INFO:teuthology.packaging:ref: None 2024-09-15T03:25:19.515 INFO:teuthology.packaging:tag: None 2024-09-15T03:25:19.515 INFO:teuthology.packaging:branch: wip-bharath9-testing-2024-09-13-1444-quincy 2024-09-15T03:25:19.515 INFO:teuthology.packaging:sha1: d75f23820806da622f561ee5ee7551015285993f 2024-09-15T03:25:19.515 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=wip-bharath9-testing-2024-09-13-1444-quincy 2024-09-15T03:25:19.671 INFO:teuthology.task.internal:Found packages for ceph version 17.2.7-1643-gd75f2382-1focal 2024-09-15T03:25:19.673 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-15T03:25:19.684 INFO:teuthology.task.internal:no buildpackages task found 2024-09-15T03:25:19.684 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-15T03:25:19.770 INFO:teuthology.task.internal:Saving configuration 2024-09-15T03:25:19.784 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-15T03:25:19.793 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-15T03:25:19.815 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi110.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905284', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-15 03:19:28.503194', 'locked_by': 'scheduled_skanta@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2024-09-15T03:25:19.833 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi177.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905284', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-15 03:19:28.501339', 'locked_by': 'scheduled_skanta@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2024-09-15T03:25:19.834 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-15T03:25:19.842 INFO:teuthology.task.internal:roles: ubuntu@smithi110.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-09-15T03:25:19.843 INFO:teuthology.task.internal:roles: ubuntu@smithi177.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-09-15T03:25:19.843 INFO:teuthology.run_tasks:Running task console_log... 2024-09-15T03:25:19.912 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f2f55f57ac0>, signals=[15]) 2024-09-15T03:25:19.913 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-15T03:25:19.923 INFO:teuthology.task.internal:Opening connections... 2024-09-15T03:25:19.923 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi110.front.sepia.ceph.com 2024-09-15T03:25:19.925 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T03:25:20.031 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi177.front.sepia.ceph.com 2024-09-15T03:25:20.032 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T03:25:20.119 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-15T03:25:20.130 DEBUG:teuthology.orchestra.run.smithi110:> uname -m 2024-09-15T03:25:20.137 INFO:teuthology.orchestra.run.smithi110.stdout:x86_64 2024-09-15T03:25:20.137 DEBUG:teuthology.orchestra.run.smithi110:> cat /etc/os-release 2024-09-15T03:25:20.187 INFO:teuthology.orchestra.run.smithi110.stdout:NAME="Ubuntu" 2024-09-15T03:25:20.187 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2024-09-15T03:25:20.187 INFO:teuthology.orchestra.run.smithi110.stdout:ID=ubuntu 2024-09-15T03:25:20.187 INFO:teuthology.orchestra.run.smithi110.stdout:ID_LIKE=debian 2024-09-15T03:25:20.187 INFO:teuthology.orchestra.run.smithi110.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2024-09-15T03:25:20.187 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION_ID="20.04" 2024-09-15T03:25:20.187 INFO:teuthology.orchestra.run.smithi110.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-15T03:25:20.187 INFO:teuthology.orchestra.run.smithi110.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-15T03:25:20.187 INFO:teuthology.orchestra.run.smithi110.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-15T03:25:20.187 INFO:teuthology.orchestra.run.smithi110.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-15T03:25:20.188 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION_CODENAME=focal 2024-09-15T03:25:20.188 INFO:teuthology.orchestra.run.smithi110.stdout:UBUNTU_CODENAME=focal 2024-09-15T03:25:20.188 INFO:teuthology.lock.ops:Updating smithi110.front.sepia.ceph.com on lock server 2024-09-15T03:25:20.207 DEBUG:teuthology.orchestra.run.smithi177:> uname -m 2024-09-15T03:25:20.214 INFO:teuthology.orchestra.run.smithi177.stdout:x86_64 2024-09-15T03:25:20.214 DEBUG:teuthology.orchestra.run.smithi177:> cat /etc/os-release 2024-09-15T03:25:20.262 INFO:teuthology.orchestra.run.smithi177.stdout:NAME="Ubuntu" 2024-09-15T03:25:20.262 INFO:teuthology.orchestra.run.smithi177.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2024-09-15T03:25:20.262 INFO:teuthology.orchestra.run.smithi177.stdout:ID=ubuntu 2024-09-15T03:25:20.262 INFO:teuthology.orchestra.run.smithi177.stdout:ID_LIKE=debian 2024-09-15T03:25:20.262 INFO:teuthology.orchestra.run.smithi177.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2024-09-15T03:25:20.262 INFO:teuthology.orchestra.run.smithi177.stdout:VERSION_ID="20.04" 2024-09-15T03:25:20.262 INFO:teuthology.orchestra.run.smithi177.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-15T03:25:20.262 INFO:teuthology.orchestra.run.smithi177.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-15T03:25:20.262 INFO:teuthology.orchestra.run.smithi177.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-15T03:25:20.262 INFO:teuthology.orchestra.run.smithi177.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-15T03:25:20.263 INFO:teuthology.orchestra.run.smithi177.stdout:VERSION_CODENAME=focal 2024-09-15T03:25:20.263 INFO:teuthology.orchestra.run.smithi177.stdout:UBUNTU_CODENAME=focal 2024-09-15T03:25:20.263 INFO:teuthology.lock.ops:Updating smithi177.front.sepia.ceph.com on lock server 2024-09-15T03:25:20.282 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-15T03:25:20.296 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-15T03:25:20.312 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-15T03:25:20.312 DEBUG:teuthology.orchestra.run.smithi110:> test '!' -e /home/ubuntu/cephtest 2024-09-15T03:25:20.318 DEBUG:teuthology.orchestra.run.smithi177:> test '!' -e /home/ubuntu/cephtest 2024-09-15T03:25:20.325 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-15T03:25:20.335 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-15T03:25:20.335 DEBUG:teuthology.orchestra.run.smithi110:> test -z $(ls -A /var/lib/ceph) 2024-09-15T03:25:20.367 DEBUG:teuthology.orchestra.run.smithi177:> test -z $(ls -A /var/lib/ceph) 2024-09-15T03:25:20.384 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-15T03:25:20.437 INFO:teuthology.run_tasks:Running task kernel... 2024-09-15T03:25:20.477 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-15T03:25:20.477 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-15T03:25:20.478 DEBUG:teuthology.orchestra.run.smithi110:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-15T03:25:20.478 DEBUG:teuthology.orchestra.run.smithi177:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-15T03:25:20.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T03:25:20.484 DEBUG:teuthology.orchestra.run.smithi110:> uname -r 2024-09-15T03:25:20.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T03:25:20.485 DEBUG:teuthology.orchestra.run.smithi177:> uname -r 2024-09-15T03:25:20.490 INFO:teuthology.orchestra.run.smithi110.stdout:5.4.0-124-generic 2024-09-15T03:25:20.491 INFO:teuthology.task.kernel:Running kernel on smithi110: 5.4.0-124-generic 2024-09-15T03:25:20.491 DEBUG:teuthology.orchestra.run.smithi110:> sudo apt-get clean 2024-09-15T03:25:20.492 INFO:teuthology.orchestra.run.smithi177.stdout:5.4.0-124-generic 2024-09-15T03:25:20.492 INFO:teuthology.task.kernel:Running kernel on smithi177: 5.4.0-124-generic 2024-09-15T03:25:20.492 DEBUG:teuthology.orchestra.run.smithi177:> sudo apt-get clean 2024-09-15T03:25:20.642 DEBUG:teuthology.orchestra.run.smithi110:> sudo apt-get update 2024-09-15T03:25:20.646 DEBUG:teuthology.orchestra.run.smithi177:> sudo apt-get update 2024-09-15T03:25:20.789 INFO:teuthology.orchestra.run.smithi110.stdout:Get:1 http://security.ubuntu.com/ubuntu focal-security InRelease [128 kB] 2024-09-15T03:25:20.806 INFO:teuthology.orchestra.run.smithi177.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-15T03:25:20.819 INFO:teuthology.orchestra.run.smithi177.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease [128 kB] 2024-09-15T03:25:20.880 INFO:teuthology.orchestra.run.smithi177.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease [128 kB] 2024-09-15T03:25:20.905 INFO:teuthology.orchestra.run.smithi110.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-15T03:25:20.915 INFO:teuthology.orchestra.run.smithi177.stdout:Get:4 http://security.ubuntu.com/ubuntu focal-security InRelease [128 kB] 2024-09-15T03:25:20.987 INFO:teuthology.orchestra.run.smithi110.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [128 kB] 2024-09-15T03:25:21.101 INFO:teuthology.orchestra.run.smithi110.stdout:Get:4 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [3,195 kB] 2024-09-15T03:25:21.202 INFO:teuthology.orchestra.run.smithi177.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [3,563 kB] 2024-09-15T03:25:21.225 INFO:teuthology.orchestra.run.smithi110.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [812 kB] 2024-09-15T03:25:21.233 INFO:teuthology.orchestra.run.smithi110.stdout:Get:6 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [472 kB] 2024-09-15T03:25:21.239 INFO:teuthology.orchestra.run.smithi110.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [14.3 kB] 2024-09-15T03:25:21.239 INFO:teuthology.orchestra.run.smithi110.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [38.3 kB] 2024-09-15T03:25:21.239 INFO:teuthology.orchestra.run.smithi110.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [3,117 kB] 2024-09-15T03:25:21.260 INFO:teuthology.orchestra.run.smithi110.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [436 kB] 2024-09-15T03:25:21.263 INFO:teuthology.orchestra.run.smithi110.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [548 B] 2024-09-15T03:25:21.269 INFO:teuthology.orchestra.run.smithi110.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [681 kB] 2024-09-15T03:25:21.274 INFO:teuthology.orchestra.run.smithi110.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [1,011 kB] 2024-09-15T03:25:21.283 INFO:teuthology.orchestra.run.smithi110.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [213 kB] 2024-09-15T03:25:21.285 INFO:teuthology.orchestra.run.smithi110.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [21.2 kB] 2024-09-15T03:25:21.285 INFO:teuthology.orchestra.run.smithi110.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [24.8 kB] 2024-09-15T03:25:21.286 INFO:teuthology.orchestra.run.smithi110.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,204 B] 2024-09-15T03:25:21.286 INFO:teuthology.orchestra.run.smithi110.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,968 B] 2024-09-15T03:25:21.286 INFO:teuthology.orchestra.run.smithi110.stdout:Get:19 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [540 B] 2024-09-15T03:25:21.331 INFO:teuthology.orchestra.run.smithi177.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [1,030 kB] 2024-09-15T03:25:21.340 INFO:teuthology.orchestra.run.smithi177.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [550 kB] 2024-09-15T03:25:21.341 INFO:teuthology.orchestra.run.smithi110.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-backports InRelease [128 kB] 2024-09-15T03:25:21.343 INFO:teuthology.orchestra.run.smithi177.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.7 kB] 2024-09-15T03:25:21.344 INFO:teuthology.orchestra.run.smithi177.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [39.6 kB] 2024-09-15T03:25:21.345 INFO:teuthology.orchestra.run.smithi177.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [3,236 kB] 2024-09-15T03:25:21.371 INFO:teuthology.orchestra.run.smithi177.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [453 kB] 2024-09-15T03:25:21.374 INFO:teuthology.orchestra.run.smithi177.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [548 B] 2024-09-15T03:25:21.374 INFO:teuthology.orchestra.run.smithi177.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [807 kB] 2024-09-15T03:25:21.379 INFO:teuthology.orchestra.run.smithi177.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,231 kB] 2024-09-15T03:25:21.393 INFO:teuthology.orchestra.run.smithi177.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [295 kB] 2024-09-15T03:25:21.395 INFO:teuthology.orchestra.run.smithi177.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [28.0 kB] 2024-09-15T03:25:21.396 INFO:teuthology.orchestra.run.smithi177.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [27.1 kB] 2024-09-15T03:25:21.413 INFO:teuthology.orchestra.run.smithi177.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,440 B] 2024-09-15T03:25:21.413 INFO:teuthology.orchestra.run.smithi177.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,936 B] 2024-09-15T03:25:21.413 INFO:teuthology.orchestra.run.smithi177.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [616 B] 2024-09-15T03:25:21.413 INFO:teuthology.orchestra.run.smithi177.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2024-09-15T03:25:21.414 INFO:teuthology.orchestra.run.smithi177.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2024-09-15T03:25:21.414 INFO:teuthology.orchestra.run.smithi177.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2024-09-15T03:25:21.424 INFO:teuthology.orchestra.run.smithi177.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2024-09-15T03:25:21.425 INFO:teuthology.orchestra.run.smithi177.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2024-09-15T03:25:21.426 INFO:teuthology.orchestra.run.smithi177.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2024-09-15T03:25:21.480 INFO:teuthology.orchestra.run.smithi110.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [3,563 kB] 2024-09-15T03:25:21.521 INFO:teuthology.orchestra.run.smithi177.stdout:Get:27 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [812 kB] 2024-09-15T03:25:21.856 INFO:teuthology.orchestra.run.smithi177.stdout:Get:28 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [3,195 kB] 2024-09-15T03:25:21.864 INFO:teuthology.orchestra.run.smithi110.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [1,030 kB] 2024-09-15T03:25:21.892 INFO:teuthology.orchestra.run.smithi110.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [550 kB] 2024-09-15T03:25:21.905 INFO:teuthology.orchestra.run.smithi110.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.7 kB] 2024-09-15T03:25:21.906 INFO:teuthology.orchestra.run.smithi110.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [39.6 kB] 2024-09-15T03:25:21.906 INFO:teuthology.orchestra.run.smithi110.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [3,236 kB] 2024-09-15T03:25:21.999 INFO:teuthology.orchestra.run.smithi110.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [453 kB] 2024-09-15T03:25:22.017 INFO:teuthology.orchestra.run.smithi110.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [548 B] 2024-09-15T03:25:22.017 INFO:teuthology.orchestra.run.smithi110.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [807 kB] 2024-09-15T03:25:22.039 INFO:teuthology.orchestra.run.smithi110.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,231 kB] 2024-09-15T03:25:22.047 INFO:teuthology.orchestra.run.smithi177.stdout:Get:29 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [472 kB] 2024-09-15T03:25:22.062 INFO:teuthology.orchestra.run.smithi177.stdout:Get:30 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [14.3 kB] 2024-09-15T03:25:22.062 INFO:teuthology.orchestra.run.smithi177.stdout:Get:31 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [3,117 kB] 2024-09-15T03:25:22.072 INFO:teuthology.orchestra.run.smithi110.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [295 kB] 2024-09-15T03:25:22.079 INFO:teuthology.orchestra.run.smithi110.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [28.0 kB] 2024-09-15T03:25:22.079 INFO:teuthology.orchestra.run.smithi110.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [27.1 kB] 2024-09-15T03:25:22.080 INFO:teuthology.orchestra.run.smithi110.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,440 B] 2024-09-15T03:25:22.080 INFO:teuthology.orchestra.run.smithi110.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,936 B] 2024-09-15T03:25:22.111 INFO:teuthology.orchestra.run.smithi110.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [616 B] 2024-09-15T03:25:22.111 INFO:teuthology.orchestra.run.smithi110.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2024-09-15T03:25:22.112 INFO:teuthology.orchestra.run.smithi110.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2024-09-15T03:25:22.153 INFO:teuthology.orchestra.run.smithi177.stdout:Get:32 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [38.3 kB] 2024-09-15T03:25:22.153 INFO:teuthology.orchestra.run.smithi177.stdout:Get:33 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [436 kB] 2024-09-15T03:25:22.166 INFO:teuthology.orchestra.run.smithi177.stdout:Get:34 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [548 B] 2024-09-15T03:25:22.166 INFO:teuthology.orchestra.run.smithi177.stdout:Get:35 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [681 kB] 2024-09-15T03:25:22.187 INFO:teuthology.orchestra.run.smithi177.stdout:Get:36 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [1,011 kB] 2024-09-15T03:25:22.192 INFO:teuthology.orchestra.run.smithi110.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2024-09-15T03:25:22.193 INFO:teuthology.orchestra.run.smithi110.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2024-09-15T03:25:22.193 INFO:teuthology.orchestra.run.smithi110.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2024-09-15T03:25:22.193 INFO:teuthology.orchestra.run.smithi110.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2024-09-15T03:25:22.217 INFO:teuthology.orchestra.run.smithi177.stdout:Get:37 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [213 kB] 2024-09-15T03:25:22.224 INFO:teuthology.orchestra.run.smithi177.stdout:Get:38 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [21.2 kB] 2024-09-15T03:25:22.224 INFO:teuthology.orchestra.run.smithi177.stdout:Get:39 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,204 B] 2024-09-15T03:25:22.225 INFO:teuthology.orchestra.run.smithi177.stdout:Get:40 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [24.8 kB] 2024-09-15T03:25:22.297 INFO:teuthology.orchestra.run.smithi177.stdout:Get:41 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,968 B] 2024-09-15T03:25:22.298 INFO:teuthology.orchestra.run.smithi177.stdout:Get:42 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [540 B] 2024-09-15T03:25:25.650 INFO:teuthology.orchestra.run.smithi110.stdout:Fetched 21.8 MB in 3s (7,408 kB/s) 2024-09-15T03:25:25.679 INFO:teuthology.orchestra.run.smithi177.stdout:Fetched 21.8 MB in 3s (7,170 kB/s) 2024-09-15T03:25:27.247 INFO:teuthology.orchestra.run.smithi110.stdout:Reading package lists... 2024-09-15T03:25:27.263 INFO:teuthology.orchestra.run.smithi177.stdout:Reading package lists... 2024-09-15T03:25:27.275 DEBUG:teuthology.orchestra.run.smithi110:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-15T03:25:27.290 DEBUG:teuthology.orchestra.run.smithi177:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-15T03:25:27.349 INFO:teuthology.orchestra.run.smithi110.stdout:Reading package lists... 2024-09-15T03:25:27.362 INFO:teuthology.orchestra.run.smithi177.stdout:Reading package lists... 2024-09-15T03:25:27.559 INFO:teuthology.orchestra.run.smithi110.stdout:Building dependency tree... 2024-09-15T03:25:27.560 INFO:teuthology.orchestra.run.smithi110.stdout:Reading state information... 2024-09-15T03:25:27.580 INFO:teuthology.orchestra.run.smithi177.stdout:Building dependency tree... 2024-09-15T03:25:27.581 INFO:teuthology.orchestra.run.smithi177.stdout:Reading state information... 2024-09-15T03:25:27.754 INFO:teuthology.orchestra.run.smithi110.stdout:The following additional packages will be installed: 2024-09-15T03:25:27.755 INFO:teuthology.orchestra.run.smithi110.stdout: linux-generic linux-headers-5.4.0-195 linux-headers-5.4.0-195-generic 2024-09-15T03:25:27.755 INFO:teuthology.orchestra.run.smithi110.stdout: linux-headers-generic linux-image-5.4.0-195-generic 2024-09-15T03:25:27.755 INFO:teuthology.orchestra.run.smithi110.stdout: linux-modules-5.4.0-195-generic linux-modules-extra-5.4.0-195-generic 2024-09-15T03:25:27.756 INFO:teuthology.orchestra.run.smithi110.stdout:Suggested packages: 2024-09-15T03:25:27.756 INFO:teuthology.orchestra.run.smithi110.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2024-09-15T03:25:27.756 INFO:teuthology.orchestra.run.smithi110.stdout:Recommended packages: 2024-09-15T03:25:27.756 INFO:teuthology.orchestra.run.smithi110.stdout: thermald 2024-09-15T03:25:27.782 INFO:teuthology.orchestra.run.smithi177.stdout:The following additional packages will be installed: 2024-09-15T03:25:27.782 INFO:teuthology.orchestra.run.smithi177.stdout: linux-generic linux-headers-5.4.0-195 linux-headers-5.4.0-195-generic 2024-09-15T03:25:27.782 INFO:teuthology.orchestra.run.smithi177.stdout: linux-headers-generic linux-image-5.4.0-195-generic 2024-09-15T03:25:27.782 INFO:teuthology.orchestra.run.smithi177.stdout: linux-modules-5.4.0-195-generic linux-modules-extra-5.4.0-195-generic 2024-09-15T03:25:27.783 INFO:teuthology.orchestra.run.smithi177.stdout:Suggested packages: 2024-09-15T03:25:27.784 INFO:teuthology.orchestra.run.smithi177.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2024-09-15T03:25:27.784 INFO:teuthology.orchestra.run.smithi177.stdout:Recommended packages: 2024-09-15T03:25:27.784 INFO:teuthology.orchestra.run.smithi177.stdout: thermald 2024-09-15T03:25:27.814 INFO:teuthology.orchestra.run.smithi110.stdout:The following NEW packages will be installed: 2024-09-15T03:25:27.814 INFO:teuthology.orchestra.run.smithi110.stdout: linux-headers-5.4.0-195 linux-headers-5.4.0-195-generic 2024-09-15T03:25:27.814 INFO:teuthology.orchestra.run.smithi110.stdout: linux-image-5.4.0-195-generic linux-modules-5.4.0-195-generic 2024-09-15T03:25:27.815 INFO:teuthology.orchestra.run.smithi110.stdout: linux-modules-extra-5.4.0-195-generic 2024-09-15T03:25:27.816 INFO:teuthology.orchestra.run.smithi110.stdout:The following packages will be upgraded: 2024-09-15T03:25:27.816 INFO:teuthology.orchestra.run.smithi110.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-15T03:25:27.842 INFO:teuthology.orchestra.run.smithi177.stdout:The following NEW packages will be installed: 2024-09-15T03:25:27.842 INFO:teuthology.orchestra.run.smithi177.stdout: linux-headers-5.4.0-195 linux-headers-5.4.0-195-generic 2024-09-15T03:25:27.842 INFO:teuthology.orchestra.run.smithi177.stdout: linux-image-5.4.0-195-generic linux-modules-5.4.0-195-generic 2024-09-15T03:25:27.843 INFO:teuthology.orchestra.run.smithi177.stdout: linux-modules-extra-5.4.0-195-generic 2024-09-15T03:25:27.844 INFO:teuthology.orchestra.run.smithi177.stdout:The following packages will be upgraded: 2024-09-15T03:25:27.844 INFO:teuthology.orchestra.run.smithi177.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-15T03:25:27.872 INFO:teuthology.orchestra.run.smithi110.stdout:3 upgraded, 5 newly installed, 0 to remove and 316 not upgraded. 2024-09-15T03:25:27.872 INFO:teuthology.orchestra.run.smithi110.stdout:Need to get 77.2 MB of archives. 2024-09-15T03:25:27.872 INFO:teuthology.orchestra.run.smithi110.stdout:After this operation, 381 MB of additional disk space will be used. 2024-09-15T03:25:27.872 INFO:teuthology.orchestra.run.smithi110.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-195-generic amd64 5.4.0-195.215 [15.0 MB] 2024-09-15T03:25:28.029 INFO:teuthology.orchestra.run.smithi177.stdout:3 upgraded, 5 newly installed, 0 to remove and 316 not upgraded. 2024-09-15T03:25:28.029 INFO:teuthology.orchestra.run.smithi177.stdout:Need to get 77.2 MB of archives. 2024-09-15T03:25:28.029 INFO:teuthology.orchestra.run.smithi177.stdout:After this operation, 381 MB of additional disk space will be used. 2024-09-15T03:25:28.030 INFO:teuthology.orchestra.run.smithi177.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-195-generic amd64 5.4.0-195.215 [15.0 MB] 2024-09-15T03:25:28.205 INFO:teuthology.orchestra.run.smithi110.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-195-generic amd64 5.4.0-195.215 [10.5 MB] 2024-09-15T03:25:28.296 INFO:teuthology.orchestra.run.smithi110.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-195-generic amd64 5.4.0-195.215 [39.2 MB] 2024-09-15T03:25:28.777 INFO:teuthology.orchestra.run.smithi110.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.195.193 [1,900 B] 2024-09-15T03:25:28.777 INFO:teuthology.orchestra.run.smithi110.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.195.193 [2,584 B] 2024-09-15T03:25:28.777 INFO:teuthology.orchestra.run.smithi110.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-195 all 5.4.0-195.215 [11.0 MB] 2024-09-15T03:25:28.871 INFO:teuthology.orchestra.run.smithi110.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-195-generic amd64 5.4.0-195.215 [1,386 kB] 2024-09-15T03:25:28.882 INFO:teuthology.orchestra.run.smithi110.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.195.193 [2,460 B] 2024-09-15T03:25:29.139 INFO:teuthology.orchestra.run.smithi177.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-195-generic amd64 5.4.0-195.215 [10.5 MB] 2024-09-15T03:25:29.408 INFO:teuthology.orchestra.run.smithi177.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-195-generic amd64 5.4.0-195.215 [39.2 MB] 2024-09-15T03:25:29.615 INFO:teuthology.orchestra.run.smithi110.stdout:Fetched 77.2 MB in 1s (72.9 MB/s) 2024-09-15T03:25:29.799 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package linux-modules-5.4.0-195-generic. 2024-09-15T03:25:30.743 INFO:teuthology.orchestra.run.smithi177.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.195.193 [1,900 B] 2024-09-15T03:25:30.743 INFO:teuthology.orchestra.run.smithi177.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.195.193 [2,584 B] 2024-09-15T03:25:30.744 INFO:teuthology.orchestra.run.smithi177.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-195 all 5.4.0-195.215 [11.0 MB] 2024-09-15T03:25:31.024 INFO:teuthology.orchestra.run.smithi177.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-195-generic amd64 5.4.0-195.215 [1,386 kB] 2024-09-15T03:25:31.046 INFO:teuthology.orchestra.run.smithi177.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.195.193 [2,460 B] 2024-09-15T03:25:31.620 INFO:teuthology.orchestra.run.smithi110.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2024-09-15T03:25:31.624 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../0-linux-modules-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-15T03:25:31.757 INFO:teuthology.orchestra.run.smithi177.stdout:Fetched 77.2 MB in 3s (24.2 MB/s) 2024-09-15T03:25:31.762 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking linux-modules-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-15T03:25:31.931 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package linux-modules-5.4.0-195-generic. 2024-09-15T03:25:33.697 INFO:teuthology.orchestra.run.smithi177.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2024-09-15T03:25:33.702 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../0-linux-modules-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-15T03:25:33.804 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package linux-image-5.4.0-195-generic. 2024-09-15T03:25:33.818 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../1-linux-image-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-15T03:25:33.828 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking linux-modules-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-15T03:25:33.918 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-15T03:25:35.040 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-195-generic. 2024-09-15T03:25:35.055 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-15T03:25:35.098 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking linux-modules-extra-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-15T03:25:35.754 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package linux-image-5.4.0-195-generic. 2024-09-15T03:25:35.769 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../1-linux-image-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-15T03:25:35.867 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-15T03:25:36.982 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-195-generic. 2024-09-15T03:25:36.997 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-15T03:25:37.040 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking linux-modules-extra-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-15T03:25:39.939 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../3-linux-generic_5.4.0.195.193_amd64.deb ... 2024-09-15T03:25:40.056 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking linux-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-15T03:25:40.400 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.195.193_amd64.deb ... 2024-09-15T03:25:40.551 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking linux-image-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-15T03:25:40.905 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package linux-headers-5.4.0-195. 2024-09-15T03:25:40.921 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../5-linux-headers-5.4.0-195_5.4.0-195.215_all.deb ... 2024-09-15T03:25:40.963 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking linux-headers-5.4.0-195 (5.4.0-195.215) ... 2024-09-15T03:25:41.821 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../3-linux-generic_5.4.0.195.193_amd64.deb ... 2024-09-15T03:25:41.946 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking linux-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-15T03:25:42.305 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.195.193_amd64.deb ... 2024-09-15T03:25:42.458 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking linux-image-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-15T03:25:42.786 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package linux-headers-5.4.0-195. 2024-09-15T03:25:42.801 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../5-linux-headers-5.4.0-195_5.4.0-195.215_all.deb ... 2024-09-15T03:25:42.844 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking linux-headers-5.4.0-195 (5.4.0-195.215) ... 2024-09-15T03:25:46.845 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package linux-headers-5.4.0-195-generic. 2024-09-15T03:25:46.867 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../6-linux-headers-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-15T03:25:46.928 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking linux-headers-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-15T03:25:48.936 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package linux-headers-5.4.0-195-generic. 2024-09-15T03:25:48.957 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../6-linux-headers-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-15T03:25:48.994 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking linux-headers-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-15T03:25:49.716 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.195.193_amd64.deb ... 2024-09-15T03:25:49.854 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking linux-headers-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-15T03:25:50.199 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up linux-modules-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-15T03:25:51.450 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-15T03:25:51.753 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.195.193_amd64.deb ... 2024-09-15T03:25:51.897 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking linux-headers-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-15T03:25:52.239 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up linux-modules-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-15T03:25:52.601 INFO:teuthology.orchestra.run.smithi110.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-195-generic 2024-09-15T03:25:52.602 INFO:teuthology.orchestra.run.smithi110.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-195-generic 2024-09-15T03:25:52.761 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up linux-modules-extra-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-15T03:25:53.611 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-15T03:25:54.912 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up linux-headers-5.4.0-195 (5.4.0-195.215) ... 2024-09-15T03:25:55.085 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up linux-image-generic (5.4.0.195.193) ... 2024-09-15T03:25:55.223 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up linux-headers-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-15T03:25:55.357 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up linux-headers-generic (5.4.0.195.193) ... 2024-09-15T03:25:55.517 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up linux-generic (5.4.0.195.193) ... 2024-09-15T03:25:55.643 INFO:teuthology.orchestra.run.smithi110.stdout:Processing triggers for linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-15T03:25:55.707 INFO:teuthology.orchestra.run.smithi110.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-15T03:25:55.707 INFO:teuthology.orchestra.run.smithi110.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-195-generic 2024-09-15T03:25:55.710 INFO:teuthology.orchestra.run.smithi177.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-195-generic 2024-09-15T03:25:55.710 INFO:teuthology.orchestra.run.smithi177.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-195-generic 2024-09-15T03:25:55.872 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up linux-modules-extra-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-15T03:25:57.074 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up linux-headers-5.4.0-195 (5.4.0-195.215) ... 2024-09-15T03:25:57.200 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up linux-image-generic (5.4.0.195.193) ... 2024-09-15T03:25:57.344 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up linux-headers-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-15T03:25:57.469 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up linux-headers-generic (5.4.0.195.193) ... 2024-09-15T03:25:57.594 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up linux-generic (5.4.0.195.193) ... 2024-09-15T03:25:57.746 INFO:teuthology.orchestra.run.smithi177.stdout:Processing triggers for linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-15T03:25:57.835 INFO:teuthology.orchestra.run.smithi177.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-15T03:25:57.835 INFO:teuthology.orchestra.run.smithi177.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-195-generic 2024-09-15T03:26:17.796 INFO:teuthology.orchestra.run.smithi110.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-15T03:26:17.797 INFO:teuthology.orchestra.run.smithi110.stdout:Sourcing file `/etc/default/grub' 2024-09-15T03:26:17.814 INFO:teuthology.orchestra.run.smithi110.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-15T03:26:17.836 INFO:teuthology.orchestra.run.smithi110.stdout:Generating grub configuration file ... 2024-09-15T03:26:18.254 INFO:teuthology.orchestra.run.smithi110.stdout:Found linux image: /boot/vmlinuz-5.4.0-195-generic 2024-09-15T03:26:18.280 INFO:teuthology.orchestra.run.smithi110.stdout:Found initrd image: /boot/initrd.img-5.4.0-195-generic 2024-09-15T03:26:18.757 INFO:teuthology.orchestra.run.smithi110.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-15T03:26:18.769 INFO:teuthology.orchestra.run.smithi110.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-15T03:26:20.053 INFO:teuthology.orchestra.run.smithi177.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-15T03:26:20.053 INFO:teuthology.orchestra.run.smithi177.stdout:Sourcing file `/etc/default/grub' 2024-09-15T03:26:20.066 INFO:teuthology.orchestra.run.smithi177.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-15T03:26:20.086 INFO:teuthology.orchestra.run.smithi177.stdout:Generating grub configuration file ... 2024-09-15T03:26:20.458 INFO:teuthology.orchestra.run.smithi177.stdout:Found linux image: /boot/vmlinuz-5.4.0-195-generic 2024-09-15T03:26:20.483 INFO:teuthology.orchestra.run.smithi177.stdout:Found initrd image: /boot/initrd.img-5.4.0-195-generic 2024-09-15T03:26:20.837 INFO:teuthology.orchestra.run.smithi110.stdout:done 2024-09-15T03:26:20.904 INFO:teuthology.orchestra.run.smithi177.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-15T03:26:20.915 INFO:teuthology.orchestra.run.smithi177.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-15T03:26:21.244 DEBUG:teuthology.orchestra.run.smithi110:> dpkg -s linux-image-generic 2024-09-15T03:26:21.293 INFO:teuthology.orchestra.run.smithi110.stdout:Package: linux-image-generic 2024-09-15T03:26:21.293 INFO:teuthology.orchestra.run.smithi110.stdout:Status: install ok installed 2024-09-15T03:26:21.293 INFO:teuthology.orchestra.run.smithi110.stdout:Priority: optional 2024-09-15T03:26:21.293 INFO:teuthology.orchestra.run.smithi110.stdout:Section: kernel 2024-09-15T03:26:21.293 INFO:teuthology.orchestra.run.smithi110.stdout:Installed-Size: 20 2024-09-15T03:26:21.293 INFO:teuthology.orchestra.run.smithi110.stdout:Maintainer: Ubuntu Kernel Team 2024-09-15T03:26:21.293 INFO:teuthology.orchestra.run.smithi110.stdout:Architecture: amd64 2024-09-15T03:26:21.293 INFO:teuthology.orchestra.run.smithi110.stdout:Source: linux-meta 2024-09-15T03:26:21.294 INFO:teuthology.orchestra.run.smithi110.stdout:Version: 5.4.0.195.193 2024-09-15T03:26:21.294 INFO:teuthology.orchestra.run.smithi110.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-15T03:26:21.294 INFO:teuthology.orchestra.run.smithi110.stdout:Depends: linux-image-5.4.0-195-generic, linux-modules-extra-5.4.0-195-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-15T03:26:21.294 INFO:teuthology.orchestra.run.smithi110.stdout:Recommends: thermald 2024-09-15T03:26:21.294 INFO:teuthology.orchestra.run.smithi110.stdout:Description: Generic Linux kernel image 2024-09-15T03:26:21.294 INFO:teuthology.orchestra.run.smithi110.stdout: This package will always depend on the latest generic kernel image 2024-09-15T03:26:21.294 INFO:teuthology.orchestra.run.smithi110.stdout: available. 2024-09-15T03:26:21.294 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-195-generic 2024-09-15T03:26:21.294 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-15T03:26:21.295 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-15T03:26:21.295 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-15T03:26:21.295 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi110.front.sepia.ceph.com, path=None, version=distro) 2024-09-15T03:26:21.295 DEBUG:teuthology.orchestra.run.smithi110:> sudo apt-get clean 2024-09-15T03:26:21.409 DEBUG:teuthology.orchestra.run.smithi110:> sudo apt-get update 2024-09-15T03:26:21.542 INFO:teuthology.orchestra.run.smithi110.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-15T03:26:21.548 INFO:teuthology.orchestra.run.smithi110.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-15T03:26:21.566 INFO:teuthology.orchestra.run.smithi110.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-15T03:26:21.661 INFO:teuthology.orchestra.run.smithi110.stdout:Hit:4 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-15T03:26:23.380 INFO:teuthology.orchestra.run.smithi110.stdout:Reading package lists... 2024-09-15T03:26:23.406 DEBUG:teuthology.orchestra.run.smithi110:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-15T03:26:23.423 INFO:teuthology.orchestra.run.smithi177.stdout:done 2024-09-15T03:26:23.473 INFO:teuthology.orchestra.run.smithi110.stdout:Reading package lists... 2024-09-15T03:26:23.692 INFO:teuthology.orchestra.run.smithi110.stdout:Building dependency tree... 2024-09-15T03:26:23.692 INFO:teuthology.orchestra.run.smithi110.stdout:Reading state information... 2024-09-15T03:26:23.838 DEBUG:teuthology.orchestra.run.smithi177:> dpkg -s linux-image-generic 2024-09-15T03:26:23.893 INFO:teuthology.orchestra.run.smithi177.stdout:Package: linux-image-generic 2024-09-15T03:26:23.893 INFO:teuthology.orchestra.run.smithi177.stdout:Status: install ok installed 2024-09-15T03:26:23.893 INFO:teuthology.orchestra.run.smithi177.stdout:Priority: optional 2024-09-15T03:26:23.893 INFO:teuthology.orchestra.run.smithi177.stdout:Section: kernel 2024-09-15T03:26:23.893 INFO:teuthology.orchestra.run.smithi177.stdout:Installed-Size: 20 2024-09-15T03:26:23.893 INFO:teuthology.orchestra.run.smithi177.stdout:Maintainer: Ubuntu Kernel Team 2024-09-15T03:26:23.893 INFO:teuthology.orchestra.run.smithi177.stdout:Architecture: amd64 2024-09-15T03:26:23.893 INFO:teuthology.orchestra.run.smithi177.stdout:Source: linux-meta 2024-09-15T03:26:23.893 INFO:teuthology.orchestra.run.smithi177.stdout:Version: 5.4.0.195.193 2024-09-15T03:26:23.893 INFO:teuthology.orchestra.run.smithi177.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-15T03:26:23.894 INFO:teuthology.orchestra.run.smithi177.stdout:Depends: linux-image-5.4.0-195-generic, linux-modules-extra-5.4.0-195-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-15T03:26:23.894 INFO:teuthology.orchestra.run.smithi177.stdout:Recommends: thermald 2024-09-15T03:26:23.894 INFO:teuthology.orchestra.run.smithi177.stdout:Description: Generic Linux kernel image 2024-09-15T03:26:23.894 INFO:teuthology.orchestra.run.smithi177.stdout: This package will always depend on the latest generic kernel image 2024-09-15T03:26:23.894 INFO:teuthology.orchestra.run.smithi177.stdout: available. 2024-09-15T03:26:23.894 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-195-generic 2024-09-15T03:26:23.894 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-15T03:26:23.894 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-15T03:26:23.895 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-15T03:26:23.895 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi177.front.sepia.ceph.com, path=None, version=distro) 2024-09-15T03:26:23.895 DEBUG:teuthology.orchestra.run.smithi177:> sudo apt-get clean 2024-09-15T03:26:23.931 INFO:teuthology.orchestra.run.smithi110.stdout:linux-image-generic is already the newest version (5.4.0.195.193). 2024-09-15T03:26:23.931 INFO:teuthology.orchestra.run.smithi110.stdout:0 upgraded, 0 newly installed, 0 to remove and 316 not upgraded. 2024-09-15T03:26:23.933 DEBUG:teuthology.orchestra.run.smithi110:> dpkg -s linux-image-generic 2024-09-15T03:26:23.954 INFO:teuthology.orchestra.run.smithi110.stdout:Package: linux-image-generic 2024-09-15T03:26:23.954 INFO:teuthology.orchestra.run.smithi110.stdout:Status: install ok installed 2024-09-15T03:26:23.954 INFO:teuthology.orchestra.run.smithi110.stdout:Priority: optional 2024-09-15T03:26:23.954 INFO:teuthology.orchestra.run.smithi110.stdout:Section: kernel 2024-09-15T03:26:23.954 INFO:teuthology.orchestra.run.smithi110.stdout:Installed-Size: 20 2024-09-15T03:26:23.954 INFO:teuthology.orchestra.run.smithi110.stdout:Maintainer: Ubuntu Kernel Team 2024-09-15T03:26:23.954 INFO:teuthology.orchestra.run.smithi110.stdout:Architecture: amd64 2024-09-15T03:26:23.954 INFO:teuthology.orchestra.run.smithi110.stdout:Source: linux-meta 2024-09-15T03:26:23.954 INFO:teuthology.orchestra.run.smithi110.stdout:Version: 5.4.0.195.193 2024-09-15T03:26:23.955 INFO:teuthology.orchestra.run.smithi110.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-15T03:26:23.955 INFO:teuthology.orchestra.run.smithi110.stdout:Depends: linux-image-5.4.0-195-generic, linux-modules-extra-5.4.0-195-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-15T03:26:23.955 INFO:teuthology.orchestra.run.smithi110.stdout:Recommends: thermald 2024-09-15T03:26:23.955 INFO:teuthology.orchestra.run.smithi110.stdout:Description: Generic Linux kernel image 2024-09-15T03:26:23.955 INFO:teuthology.orchestra.run.smithi110.stdout: This package will always depend on the latest generic kernel image 2024-09-15T03:26:23.955 INFO:teuthology.orchestra.run.smithi110.stdout: available. 2024-09-15T03:26:23.955 DEBUG:teuthology.orchestra.run.smithi110:> mktemp 2024-09-15T03:26:23.961 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/tmp.BsKG3g8HKK 2024-09-15T03:26:23.961 DEBUG:teuthology.orchestra.run.smithi110:> sudo cp /boot/grub/grub.cfg /tmp/tmp.BsKG3g8HKK 2024-09-15T03:26:23.977 DEBUG:teuthology.orchestra.run.smithi177:> sudo apt-get update 2024-09-15T03:26:24.017 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod 0666 /tmp/tmp.BsKG3g8HKK 2024-09-15T03:26:24.054 DEBUG:teuthology.orchestra.remote:smithi110:/tmp/tmp.BsKG3g8HKK is 9KB 2024-09-15T03:26:24.067 DEBUG:teuthology.orchestra.run.smithi110:> rm -fr /tmp/tmp.BsKG3g8HKK 2024-09-15T03:26:24.073 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-15T03:26:24.103 INFO:teuthology.orchestra.run.smithi177.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-15T03:26:24.111 INFO:teuthology.orchestra.run.smithi177.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-15T03:26:24.128 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-15T03:26:24.128 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-15T03:26:24.128 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-15T03:26:24.128 INFO:teuthology.orchestra.run.smithi177.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-15T03:26:24.163 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-195-generic 2024-09-15T03:26:24.163 DEBUG:teuthology.orchestra.run.smithi110:> sudo update-grub 2024-09-15T03:26:24.224 INFO:teuthology.orchestra.run.smithi177.stdout:Hit:4 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-15T03:26:25.521 INFO:teuthology.orchestra.run.smithi110.stderr:Sourcing file `/etc/default/grub' 2024-09-15T03:26:25.529 INFO:teuthology.orchestra.run.smithi110.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-15T03:26:25.577 INFO:teuthology.orchestra.run.smithi110.stderr:Generating grub configuration file ... 2024-09-15T03:26:25.894 INFO:teuthology.orchestra.run.smithi110.stderr:Found linux image: /boot/vmlinuz-5.4.0-195-generic 2024-09-15T03:26:25.906 INFO:teuthology.orchestra.run.smithi110.stderr:Found initrd image: /boot/initrd.img-5.4.0-195-generic 2024-09-15T03:26:25.953 INFO:teuthology.orchestra.run.smithi177.stdout:Reading package lists... 2024-09-15T03:26:25.981 DEBUG:teuthology.orchestra.run.smithi177:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-15T03:26:26.052 INFO:teuthology.orchestra.run.smithi177.stdout:Reading package lists... 2024-09-15T03:26:26.247 INFO:teuthology.orchestra.run.smithi177.stdout:Building dependency tree... 2024-09-15T03:26:26.247 INFO:teuthology.orchestra.run.smithi177.stdout:Reading state information... 2024-09-15T03:26:26.344 INFO:teuthology.orchestra.run.smithi110.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-15T03:26:26.355 INFO:teuthology.orchestra.run.smithi110.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-15T03:26:26.482 INFO:teuthology.orchestra.run.smithi177.stdout:linux-image-generic is already the newest version (5.4.0.195.193). 2024-09-15T03:26:26.482 INFO:teuthology.orchestra.run.smithi177.stdout:0 upgraded, 0 newly installed, 0 to remove and 316 not upgraded. 2024-09-15T03:26:26.484 DEBUG:teuthology.orchestra.run.smithi177:> dpkg -s linux-image-generic 2024-09-15T03:26:26.503 INFO:teuthology.orchestra.run.smithi177.stdout:Package: linux-image-generic 2024-09-15T03:26:26.503 INFO:teuthology.orchestra.run.smithi177.stdout:Status: install ok installed 2024-09-15T03:26:26.503 INFO:teuthology.orchestra.run.smithi177.stdout:Priority: optional 2024-09-15T03:26:26.503 INFO:teuthology.orchestra.run.smithi177.stdout:Section: kernel 2024-09-15T03:26:26.504 INFO:teuthology.orchestra.run.smithi177.stdout:Installed-Size: 20 2024-09-15T03:26:26.504 INFO:teuthology.orchestra.run.smithi177.stdout:Maintainer: Ubuntu Kernel Team 2024-09-15T03:26:26.504 INFO:teuthology.orchestra.run.smithi177.stdout:Architecture: amd64 2024-09-15T03:26:26.504 INFO:teuthology.orchestra.run.smithi177.stdout:Source: linux-meta 2024-09-15T03:26:26.504 INFO:teuthology.orchestra.run.smithi177.stdout:Version: 5.4.0.195.193 2024-09-15T03:26:26.504 INFO:teuthology.orchestra.run.smithi177.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-15T03:26:26.504 INFO:teuthology.orchestra.run.smithi177.stdout:Depends: linux-image-5.4.0-195-generic, linux-modules-extra-5.4.0-195-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-15T03:26:26.504 INFO:teuthology.orchestra.run.smithi177.stdout:Recommends: thermald 2024-09-15T03:26:26.504 INFO:teuthology.orchestra.run.smithi177.stdout:Description: Generic Linux kernel image 2024-09-15T03:26:26.504 INFO:teuthology.orchestra.run.smithi177.stdout: This package will always depend on the latest generic kernel image 2024-09-15T03:26:26.504 INFO:teuthology.orchestra.run.smithi177.stdout: available. 2024-09-15T03:26:26.505 DEBUG:teuthology.orchestra.run.smithi177:> mktemp 2024-09-15T03:26:26.557 INFO:teuthology.orchestra.run.smithi177.stdout:/tmp/tmp.0ZNJCA7kna 2024-09-15T03:26:26.557 DEBUG:teuthology.orchestra.run.smithi177:> sudo cp /boot/grub/grub.cfg /tmp/tmp.0ZNJCA7kna 2024-09-15T03:26:26.581 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 0666 /tmp/tmp.0ZNJCA7kna 2024-09-15T03:26:26.666 DEBUG:teuthology.orchestra.remote:smithi177:/tmp/tmp.0ZNJCA7kna is 9KB 2024-09-15T03:26:26.681 DEBUG:teuthology.orchestra.run.smithi177:> rm -fr /tmp/tmp.0ZNJCA7kna 2024-09-15T03:26:26.687 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-15T03:26:26.743 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-15T03:26:26.743 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-15T03:26:26.743 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-15T03:26:26.780 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-195-generic 2024-09-15T03:26:26.781 DEBUG:teuthology.orchestra.run.smithi177:> sudo update-grub 2024-09-15T03:26:27.282 INFO:teuthology.orchestra.run.smithi110.stderr:done 2024-09-15T03:26:27.283 DEBUG:teuthology.orchestra.run.smithi110:> sudo shutdown -r now 2024-09-15T03:26:28.165 INFO:teuthology.orchestra.run.smithi177.stderr:Sourcing file `/etc/default/grub' 2024-09-15T03:26:28.174 INFO:teuthology.orchestra.run.smithi177.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-15T03:26:28.184 INFO:teuthology.orchestra.run.smithi177.stderr:Generating grub configuration file ... 2024-09-15T03:26:28.434 INFO:teuthology.orchestra.run.smithi177.stderr:Found linux image: /boot/vmlinuz-5.4.0-195-generic 2024-09-15T03:26:28.446 INFO:teuthology.orchestra.run.smithi177.stderr:Found initrd image: /boot/initrd.img-5.4.0-195-generic 2024-09-15T03:26:28.833 INFO:teuthology.orchestra.run.smithi177.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-15T03:26:28.844 INFO:teuthology.orchestra.run.smithi177.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-15T03:26:30.234 INFO:teuthology.orchestra.run.smithi177.stderr:done 2024-09-15T03:26:30.236 DEBUG:teuthology.orchestra.run.smithi177:> sudo shutdown -r now 2024-09-15T03:26:57.313 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-15T03:26:57.314 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-09-15T03:26:57.315 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T03:27:00.241 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-15T03:27:00.241 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi177.front.sepia.ceph.com' 2024-09-15T03:27:00.242 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T03:27:34.722 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.177 2024-09-15T03:27:43.731 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi177.front.sepia.ceph.com' 2024-09-15T03:27:43.733 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T03:27:57.331 DEBUG:teuthology.orchestra.remote:timed out 2024-09-15T03:28:02.113 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.177 2024-09-15T03:28:06.335 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-09-15T03:28:06.336 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T03:28:06.601 DEBUG:teuthology.orchestra.run.smithi110:> true 2024-09-15T03:28:07.171 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-09-15T03:28:07.171 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.4.0-195-generic"... 2024-09-15T03:28:07.172 DEBUG:teuthology.orchestra.run.smithi110:> uname -r 2024-09-15T03:28:07.180 INFO:teuthology.orchestra.run.smithi110.stdout:5.4.0-195-generic 2024-09-15T03:28:07.180 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-195-generic vs 5.4.0-195-generic 2024-09-15T03:28:07.180 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-15T03:28:07.180 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-15T03:28:08.181 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-15T03:28:08.181 DEBUG:teuthology.orchestra.run.smithi110:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-15T03:28:08.260 INFO:teuthology.orchestra.run.smithi110.stdout:ttyS1 2024-09-15T03:28:08.282 DEBUG:teuthology.parallel:result is None 2024-09-15T03:28:14.113 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi177.front.sepia.ceph.com' 2024-09-15T03:28:14.114 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T03:28:14.359 DEBUG:teuthology.orchestra.run.smithi177:> true 2024-09-15T03:28:14.974 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi177.front.sepia.ceph.com' 2024-09-15T03:28:14.974 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.4.0-195-generic"... 2024-09-15T03:28:14.975 DEBUG:teuthology.orchestra.run.smithi177:> uname -r 2024-09-15T03:28:14.981 INFO:teuthology.orchestra.run.smithi177.stdout:5.4.0-195-generic 2024-09-15T03:28:14.981 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-195-generic vs 5.4.0-195-generic 2024-09-15T03:28:14.981 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-15T03:28:14.981 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-15T03:28:15.982 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-15T03:28:15.982 DEBUG:teuthology.orchestra.run.smithi177:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-15T03:28:16.098 INFO:teuthology.orchestra.run.smithi177.stdout:ttyS1 2024-09-15T03:28:16.118 DEBUG:teuthology.parallel:result is None 2024-09-15T03:28:16.118 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-15T03:28:16.125 INFO:teuthology.task.internal:Creating test directory... 2024-09-15T03:28:16.125 DEBUG:teuthology.orchestra.run.smithi110:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-15T03:28:16.127 DEBUG:teuthology.orchestra.run.smithi177:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-15T03:28:16.134 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-15T03:28:16.141 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-15T03:28:16.147 INFO:teuthology.task.internal:Creating archive directory... 2024-09-15T03:28:16.147 DEBUG:teuthology.orchestra.run.smithi110:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-15T03:28:16.177 DEBUG:teuthology.orchestra.run.smithi177:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-15T03:28:16.200 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-15T03:28:16.210 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-15T03:28:16.211 DEBUG:teuthology.orchestra.run.smithi110:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-15T03:28:16.214 DEBUG:teuthology.orchestra.run.smithi177:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-15T03:28:16.230 INFO:teuthology.orchestra.run.smithi110.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T03:28:16.233 INFO:teuthology.orchestra.run.smithi177.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T03:28:16.241 INFO:teuthology.orchestra.run.smithi110.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T03:28:16.243 INFO:teuthology.orchestra.run.smithi177.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T03:28:16.244 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-15T03:28:16.251 INFO:teuthology.task.internal:Configuring sudo... 2024-09-15T03:28:16.251 DEBUG:teuthology.orchestra.run.smithi110:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-15T03:28:16.253 DEBUG:teuthology.orchestra.run.smithi177:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-15T03:28:16.305 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-15T03:28:16.315 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-15T03:28:16.315 DEBUG:teuthology.orchestra.run.smithi110:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-15T03:28:16.317 DEBUG:teuthology.orchestra.run.smithi177:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-15T03:28:16.323 DEBUG:teuthology.orchestra.run.smithi110:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T03:28:16.330 DEBUG:teuthology.orchestra.run.smithi110:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T03:28:16.337 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-15T03:28:16.337 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-15T03:28:16.407 DEBUG:teuthology.orchestra.run.smithi177:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T03:28:16.414 DEBUG:teuthology.orchestra.run.smithi177:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T03:28:16.421 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-15T03:28:16.421 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-15T03:28:16.443 DEBUG:teuthology.orchestra.run.smithi110:> sudo service rsyslog restart 2024-09-15T03:28:16.445 DEBUG:teuthology.orchestra.run.smithi177:> sudo service rsyslog restart 2024-09-15T03:28:16.493 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-15T03:28:16.500 INFO:teuthology.task.internal:Starting timer... 2024-09-15T03:28:16.500 INFO:teuthology.run_tasks:Running task pcp... 2024-09-15T03:28:16.511 INFO:teuthology.run_tasks:Running task selinux... 2024-09-15T03:28:16.520 DEBUG:teuthology.task.selinux:Excluding smithi110: OS 'ubuntu' does not support SELinux 2024-09-15T03:28:16.520 DEBUG:teuthology.task.selinux:Excluding smithi177: OS 'ubuntu' does not support SELinux 2024-09-15T03:28:16.520 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-15T03:28:16.520 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-15T03:28:16.520 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-15T03:28:16.520 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-15T03:28:16.528 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-15T03:28:16.530 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-15T03:28:16.646 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-15T03:28:16.657 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-15T03:28:16.658 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi110.front.sepia.ceph.com,smithi177.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-15T03:33:47.920 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi110.front.sepia.ceph.com'), Remote(name='ubuntu@smithi177.front.sepia.ceph.com')] 2024-09-15T03:33:47.922 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-09-15T03:33:47.923 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T03:33:48.023 DEBUG:teuthology.orchestra.run.smithi110:> true 2024-09-15T03:33:48.146 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-09-15T03:33:48.147 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi177.front.sepia.ceph.com' 2024-09-15T03:33:48.147 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T03:33:48.237 DEBUG:teuthology.orchestra.run.smithi177:> true 2024-09-15T03:33:48.366 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi177.front.sepia.ceph.com' 2024-09-15T03:33:48.366 INFO:teuthology.run_tasks:Running task clock... 2024-09-15T03:33:48.375 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-15T03:33:48.375 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-15T03:33:48.376 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T03:33:48.378 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-15T03:33:48.378 DEBUG:teuthology.orchestra.run.smithi177:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T03:33:48.408 INFO:teuthology.orchestra.run.smithi110.stdout:15 Sep 03:33:48 ntpd[13739]: ntpd 4.2.8p12@1.3728-o (1): Starting 2024-09-15T03:33:48.408 INFO:teuthology.orchestra.run.smithi110.stdout:15 Sep 03:33:48 ntpd[13739]: Command line: ntpd -gq 2024-09-15T03:33:48.409 INFO:teuthology.orchestra.run.smithi110.stdout:15 Sep 03:33:48 ntpd[13739]: proto: precision = 0.158 usec (-23) 2024-09-15T03:33:48.409 INFO:teuthology.orchestra.run.smithi110.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-15T03:33:48.409 INFO:teuthology.orchestra.run.smithi110.stdout:15 Sep 03:33:48 ntpd[13739]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-15T03:33:48.410 INFO:teuthology.orchestra.run.smithi110.stdout:15 Sep 03:33:48 ntpd[13739]: restrict ::: KOD does nothing without LIMITED. 2024-09-15T03:33:48.410 INFO:teuthology.orchestra.run.smithi110.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-15T03:33:48.411 INFO:teuthology.orchestra.run.smithi110.stdout:15 Sep 03:33:48 ntpd[13739]: Listen and drop on 0 v6wildcard [::]:123 2024-09-15T03:33:48.411 INFO:teuthology.orchestra.run.smithi110.stdout:15 Sep 03:33:48 ntpd[13739]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-15T03:33:48.411 INFO:teuthology.orchestra.run.smithi110.stdout:15 Sep 03:33:48 ntpd[13739]: Listen normally on 2 lo 127.0.0.1:123 2024-09-15T03:33:48.411 INFO:teuthology.orchestra.run.smithi110.stdout:15 Sep 03:33:48 ntpd[13739]: Listen normally on 3 enp3s0f1 172.21.15.110:123 2024-09-15T03:33:48.411 INFO:teuthology.orchestra.run.smithi110.stdout:15 Sep 03:33:48 ntpd[13739]: Listen normally on 4 lo [::1]:123 2024-09-15T03:33:48.411 INFO:teuthology.orchestra.run.smithi110.stdout:15 Sep 03:33:48 ntpd[13739]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:cc35%5]:123 2024-09-15T03:33:48.411 INFO:teuthology.orchestra.run.smithi110.stdout:15 Sep 03:33:48 ntpd[13739]: Listening on routing socket on fd #22 for interface updates 2024-09-15T03:33:48.412 INFO:teuthology.orchestra.run.smithi177.stdout:15 Sep 03:33:48 ntpd[13744]: ntpd 4.2.8p12@1.3728-o (1): Starting 2024-09-15T03:33:48.412 INFO:teuthology.orchestra.run.smithi177.stdout:15 Sep 03:33:48 ntpd[13744]: Command line: ntpd -gq 2024-09-15T03:33:48.413 INFO:teuthology.orchestra.run.smithi177.stdout:15 Sep 03:33:48 ntpd[13744]: proto: precision = 0.116 usec (-23) 2024-09-15T03:33:48.413 INFO:teuthology.orchestra.run.smithi177.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-15T03:33:48.414 INFO:teuthology.orchestra.run.smithi177.stdout:15 Sep 03:33:48 ntpd[13744]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-15T03:33:48.414 INFO:teuthology.orchestra.run.smithi177.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-15T03:33:48.416 INFO:teuthology.orchestra.run.smithi177.stdout:15 Sep 03:33:48 ntpd[13744]: restrict ::: KOD does nothing without LIMITED. 2024-09-15T03:33:48.416 INFO:teuthology.orchestra.run.smithi177.stdout:15 Sep 03:33:48 ntpd[13744]: Listen and drop on 0 v6wildcard [::]:123 2024-09-15T03:33:48.417 INFO:teuthology.orchestra.run.smithi177.stdout:15 Sep 03:33:48 ntpd[13744]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-15T03:33:48.417 INFO:teuthology.orchestra.run.smithi177.stdout:15 Sep 03:33:48 ntpd[13744]: Listen normally on 2 lo 127.0.0.1:123 2024-09-15T03:33:48.417 INFO:teuthology.orchestra.run.smithi177.stdout:15 Sep 03:33:48 ntpd[13744]: Listen normally on 3 enp3s0f1 172.21.15.177:123 2024-09-15T03:33:48.417 INFO:teuthology.orchestra.run.smithi177.stdout:15 Sep 03:33:48 ntpd[13744]: Listen normally on 4 lo [::1]:123 2024-09-15T03:33:48.417 INFO:teuthology.orchestra.run.smithi177.stdout:15 Sep 03:33:48 ntpd[13744]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:feea:b04b%5]:123 2024-09-15T03:33:48.417 INFO:teuthology.orchestra.run.smithi177.stdout:15 Sep 03:33:48 ntpd[13744]: Listening on routing socket on fd #22 for interface updates 2024-09-15T03:33:49.409 INFO:teuthology.orchestra.run.smithi110.stderr:15 Sep 03:33:49 ntpd[13739]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:33:49.409 INFO:teuthology.orchestra.run.smithi110.stderr:15 Sep 03:33:49 ntpd[13739]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:33:49.409 INFO:teuthology.orchestra.run.smithi110.stderr:15 Sep 03:33:49 ntpd[13739]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:33:49.409 INFO:teuthology.orchestra.run.smithi110.stderr:15 Sep 03:33:49 ntpd[13739]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:33:49.409 INFO:teuthology.orchestra.run.smithi110.stderr:15 Sep 03:33:49 ntpd[13739]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T03:33:49.409 INFO:teuthology.orchestra.run.smithi110.stderr:15 Sep 03:33:49 ntpd[13739]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T03:33:50.408 INFO:teuthology.orchestra.run.smithi110.stderr:15 Sep 03:33:50 ntpd[13739]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:33:50.409 INFO:teuthology.orchestra.run.smithi110.stderr:15 Sep 03:33:50 ntpd[13739]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:33:50.412 INFO:teuthology.orchestra.run.smithi177.stderr:15 Sep 03:33:50 ntpd[13744]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:33:50.413 INFO:teuthology.orchestra.run.smithi177.stderr:15 Sep 03:33:50 ntpd[13744]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:33:50.413 INFO:teuthology.orchestra.run.smithi177.stderr:15 Sep 03:33:50 ntpd[13744]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:33:50.413 INFO:teuthology.orchestra.run.smithi177.stderr:15 Sep 03:33:50 ntpd[13744]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:33:50.413 INFO:teuthology.orchestra.run.smithi177.stderr:15 Sep 03:33:50 ntpd[13744]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:33:50.413 INFO:teuthology.orchestra.run.smithi177.stderr:15 Sep 03:33:50 ntpd[13744]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:33:50.413 INFO:teuthology.orchestra.run.smithi177.stderr:15 Sep 03:33:50 ntpd[13744]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T03:33:50.413 INFO:teuthology.orchestra.run.smithi177.stderr:15 Sep 03:33:50 ntpd[13744]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T03:33:51.409 INFO:teuthology.orchestra.run.smithi110.stderr:15 Sep 03:33:51 ntpd[13739]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:33:51.409 INFO:teuthology.orchestra.run.smithi110.stderr:15 Sep 03:33:51 ntpd[13739]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:33:51.409 INFO:teuthology.orchestra.run.smithi110.stderr:15 Sep 03:33:51 ntpd[13739]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:33:51.409 INFO:teuthology.orchestra.run.smithi110.stderr:15 Sep 03:33:51 ntpd[13739]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:33:51.409 INFO:teuthology.orchestra.run.smithi110.stderr:15 Sep 03:33:51 ntpd[13739]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T03:33:51.410 INFO:teuthology.orchestra.run.smithi110.stderr:15 Sep 03:33:51 ntpd[13739]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T03:33:51.412 INFO:teuthology.orchestra.run.smithi177.stderr:15 Sep 03:33:51 ntpd[13744]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:33:51.412 INFO:teuthology.orchestra.run.smithi177.stderr:15 Sep 03:33:51 ntpd[13744]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:33:51.412 INFO:teuthology.orchestra.run.smithi177.stderr:15 Sep 03:33:51 ntpd[13744]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:33:51.412 INFO:teuthology.orchestra.run.smithi177.stderr:15 Sep 03:33:51 ntpd[13744]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:33:51.412 INFO:teuthology.orchestra.run.smithi177.stderr:15 Sep 03:33:51 ntpd[13744]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T03:33:51.413 INFO:teuthology.orchestra.run.smithi177.stderr:15 Sep 03:33:51 ntpd[13744]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T03:33:52.412 INFO:teuthology.orchestra.run.smithi177.stderr:15 Sep 03:33:52 ntpd[13744]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:33:52.412 INFO:teuthology.orchestra.run.smithi177.stderr:15 Sep 03:33:52 ntpd[13744]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:33:52.412 INFO:teuthology.orchestra.run.smithi177.stderr:15 Sep 03:33:52 ntpd[13744]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:33:52.413 INFO:teuthology.orchestra.run.smithi177.stderr:15 Sep 03:33:52 ntpd[13744]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:33:52.413 INFO:teuthology.orchestra.run.smithi177.stderr:15 Sep 03:33:52 ntpd[13744]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T03:33:52.413 INFO:teuthology.orchestra.run.smithi177.stderr:15 Sep 03:33:52 ntpd[13744]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T03:33:53.410 INFO:teuthology.orchestra.run.smithi110.stderr:15 Sep 03:33:53 ntpd[13739]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:33:53.410 INFO:teuthology.orchestra.run.smithi110.stderr:15 Sep 03:33:53 ntpd[13739]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:33:53.410 INFO:teuthology.orchestra.run.smithi110.stderr:15 Sep 03:33:53 ntpd[13739]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:33:53.410 INFO:teuthology.orchestra.run.smithi110.stderr:15 Sep 03:33:53 ntpd[13739]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:33:53.410 INFO:teuthology.orchestra.run.smithi110.stderr:15 Sep 03:33:53 ntpd[13739]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:33:53.410 INFO:teuthology.orchestra.run.smithi110.stderr:15 Sep 03:33:53 ntpd[13739]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:33:53.411 INFO:teuthology.orchestra.run.smithi110.stderr:15 Sep 03:33:53 ntpd[13739]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:33:53.411 INFO:teuthology.orchestra.run.smithi110.stderr:15 Sep 03:33:53 ntpd[13739]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:33:53.411 INFO:teuthology.orchestra.run.smithi110.stderr:15 Sep 03:33:53 ntpd[13739]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T03:33:53.411 INFO:teuthology.orchestra.run.smithi110.stderr:15 Sep 03:33:53 ntpd[13739]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T03:33:53.411 INFO:teuthology.orchestra.run.smithi110.stderr:15 Sep 03:33:53 ntpd[13739]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:33:53.411 INFO:teuthology.orchestra.run.smithi110.stderr:15 Sep 03:33:53 ntpd[13739]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:33:53.411 INFO:teuthology.orchestra.run.smithi110.stderr:15 Sep 03:33:53 ntpd[13739]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T03:33:53.411 INFO:teuthology.orchestra.run.smithi110.stderr:15 Sep 03:33:53 ntpd[13739]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T03:33:53.411 INFO:teuthology.orchestra.run.smithi110.stderr:15 Sep 03:33:53 ntpd[13739]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:33:53.411 INFO:teuthology.orchestra.run.smithi110.stderr:15 Sep 03:33:53 ntpd[13739]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:33:53.411 INFO:teuthology.orchestra.run.smithi110.stderr:15 Sep 03:33:53 ntpd[13739]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T03:33:53.411 INFO:teuthology.orchestra.run.smithi110.stderr:15 Sep 03:33:53 ntpd[13739]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T03:33:53.413 INFO:teuthology.orchestra.run.smithi177.stderr:15 Sep 03:33:53 ntpd[13744]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:33:53.413 INFO:teuthology.orchestra.run.smithi177.stderr:15 Sep 03:33:53 ntpd[13744]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:33:53.413 INFO:teuthology.orchestra.run.smithi177.stderr:15 Sep 03:33:53 ntpd[13744]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:33:53.413 INFO:teuthology.orchestra.run.smithi177.stderr:15 Sep 03:33:53 ntpd[13744]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:33:53.413 INFO:teuthology.orchestra.run.smithi177.stderr:15 Sep 03:33:53 ntpd[13744]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:33:53.413 INFO:teuthology.orchestra.run.smithi177.stderr:15 Sep 03:33:53 ntpd[13744]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:33:53.413 INFO:teuthology.orchestra.run.smithi177.stderr:15 Sep 03:33:53 ntpd[13744]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T03:33:53.413 INFO:teuthology.orchestra.run.smithi177.stderr:15 Sep 03:33:53 ntpd[13744]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T03:33:53.413 INFO:teuthology.orchestra.run.smithi177.stderr:15 Sep 03:33:53 ntpd[13744]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:33:53.414 INFO:teuthology.orchestra.run.smithi177.stderr:15 Sep 03:33:53 ntpd[13744]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:33:53.414 INFO:teuthology.orchestra.run.smithi177.stderr:15 Sep 03:33:53 ntpd[13744]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T03:33:53.414 INFO:teuthology.orchestra.run.smithi177.stderr:15 Sep 03:33:53 ntpd[13744]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T03:33:54.412 INFO:teuthology.orchestra.run.smithi177.stderr:15 Sep 03:33:54 ntpd[13744]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:33:54.412 INFO:teuthology.orchestra.run.smithi177.stderr:15 Sep 03:33:54 ntpd[13744]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:33:54.412 INFO:teuthology.orchestra.run.smithi177.stderr:15 Sep 03:33:54 ntpd[13744]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:33:54.413 INFO:teuthology.orchestra.run.smithi177.stderr:15 Sep 03:33:54 ntpd[13744]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:33:54.413 INFO:teuthology.orchestra.run.smithi177.stderr:15 Sep 03:33:54 ntpd[13744]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T03:33:54.413 INFO:teuthology.orchestra.run.smithi177.stderr:15 Sep 03:33:54 ntpd[13744]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T03:33:55.410 INFO:teuthology.orchestra.run.smithi110.stderr:15 Sep 03:33:55 ntpd[13739]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:33:55.410 INFO:teuthology.orchestra.run.smithi110.stderr:15 Sep 03:33:55 ntpd[13739]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:33:55.410 INFO:teuthology.orchestra.run.smithi110.stderr:15 Sep 03:33:55 ntpd[13739]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:33:55.411 INFO:teuthology.orchestra.run.smithi110.stderr:15 Sep 03:33:55 ntpd[13739]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:33:55.411 INFO:teuthology.orchestra.run.smithi110.stderr:15 Sep 03:33:55 ntpd[13739]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:33:55.411 INFO:teuthology.orchestra.run.smithi110.stderr:15 Sep 03:33:55 ntpd[13739]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:33:55.411 INFO:teuthology.orchestra.run.smithi110.stderr:15 Sep 03:33:55 ntpd[13739]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:33:55.411 INFO:teuthology.orchestra.run.smithi110.stderr:15 Sep 03:33:55 ntpd[13739]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:33:55.411 INFO:teuthology.orchestra.run.smithi110.stderr:15 Sep 03:33:55 ntpd[13739]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T03:33:55.411 INFO:teuthology.orchestra.run.smithi110.stderr:15 Sep 03:33:55 ntpd[13739]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T03:33:55.411 INFO:teuthology.orchestra.run.smithi110.stderr:15 Sep 03:33:55 ntpd[13739]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:33:55.411 INFO:teuthology.orchestra.run.smithi110.stderr:15 Sep 03:33:55 ntpd[13739]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:33:55.411 INFO:teuthology.orchestra.run.smithi110.stderr:15 Sep 03:33:55 ntpd[13739]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T03:33:55.411 INFO:teuthology.orchestra.run.smithi110.stderr:15 Sep 03:33:55 ntpd[13739]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T03:33:55.411 INFO:teuthology.orchestra.run.smithi110.stderr:15 Sep 03:33:55 ntpd[13739]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:33:55.412 INFO:teuthology.orchestra.run.smithi110.stderr:15 Sep 03:33:55 ntpd[13739]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:33:55.412 INFO:teuthology.orchestra.run.smithi110.stderr:15 Sep 03:33:55 ntpd[13739]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T03:33:55.412 INFO:teuthology.orchestra.run.smithi110.stderr:15 Sep 03:33:55 ntpd[13739]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T03:33:55.412 INFO:teuthology.orchestra.run.smithi110.stdout:15 Sep 03:33:55 ntpd[13739]: ntpd: time slew +0.000909 s 2024-09-15T03:33:55.412 INFO:teuthology.orchestra.run.smithi110.stdout:ntpd: time slew +0.000909s 2024-09-15T03:33:55.413 INFO:teuthology.orchestra.run.smithi110.stderr:15 Sep 03:33:55 ntpd[13739]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-15T03:33:55.413 INFO:teuthology.orchestra.run.smithi110.stderr:15 Sep 03:33:55 ntpd[13739]: can't open /var/log/ntpstats/loopstats.20240915: Permission denied 2024-09-15T03:33:55.414 INFO:teuthology.orchestra.run.smithi177.stderr:15 Sep 03:33:55 ntpd[13744]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:33:55.414 INFO:teuthology.orchestra.run.smithi177.stderr:15 Sep 03:33:55 ntpd[13744]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:33:55.414 INFO:teuthology.orchestra.run.smithi177.stderr:15 Sep 03:33:55 ntpd[13744]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:33:55.414 INFO:teuthology.orchestra.run.smithi177.stderr:15 Sep 03:33:55 ntpd[13744]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:33:55.414 INFO:teuthology.orchestra.run.smithi177.stderr:15 Sep 03:33:55 ntpd[13744]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:33:55.415 INFO:teuthology.orchestra.run.smithi177.stderr:15 Sep 03:33:55 ntpd[13744]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:33:55.415 INFO:teuthology.orchestra.run.smithi177.stderr:15 Sep 03:33:55 ntpd[13744]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T03:33:55.415 INFO:teuthology.orchestra.run.smithi177.stderr:15 Sep 03:33:55 ntpd[13744]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T03:33:55.415 INFO:teuthology.orchestra.run.smithi177.stderr:15 Sep 03:33:55 ntpd[13744]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:33:55.415 INFO:teuthology.orchestra.run.smithi177.stderr:15 Sep 03:33:55 ntpd[13744]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:33:55.415 INFO:teuthology.orchestra.run.smithi177.stderr:15 Sep 03:33:55 ntpd[13744]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T03:33:55.415 INFO:teuthology.orchestra.run.smithi177.stderr:15 Sep 03:33:55 ntpd[13744]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T03:33:55.467 INFO:teuthology.orchestra.run.smithi110.stdout: remote refid st t when poll reach delay offset jitter 2024-09-15T03:33:55.467 INFO:teuthology.orchestra.run.smithi110.stdout:============================================================================== 2024-09-15T03:33:55.468 INFO:teuthology.orchestra.run.smithi110.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-15T03:33:55.468 INFO:teuthology.orchestra.run.smithi110.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-15T03:33:55.468 INFO:teuthology.orchestra.run.smithi110.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-15T03:33:55.468 INFO:teuthology.orchestra.run.smithi110.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-15T03:33:56.412 INFO:teuthology.orchestra.run.smithi177.stderr:15 Sep 03:33:56 ntpd[13744]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:33:56.412 INFO:teuthology.orchestra.run.smithi177.stderr:15 Sep 03:33:56 ntpd[13744]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:33:56.412 INFO:teuthology.orchestra.run.smithi177.stderr:15 Sep 03:33:56 ntpd[13744]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:33:56.413 INFO:teuthology.orchestra.run.smithi177.stderr:15 Sep 03:33:56 ntpd[13744]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:33:56.413 INFO:teuthology.orchestra.run.smithi177.stderr:15 Sep 03:33:56 ntpd[13744]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T03:33:56.413 INFO:teuthology.orchestra.run.smithi177.stderr:15 Sep 03:33:56 ntpd[13744]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T03:33:56.413 INFO:teuthology.orchestra.run.smithi177.stdout:15 Sep 03:33:56 ntpd[13744]: ntpd: time slew -0.000970 s 2024-09-15T03:33:56.413 INFO:teuthology.orchestra.run.smithi177.stdout:ntpd: time slew -0.000970s 2024-09-15T03:33:56.414 INFO:teuthology.orchestra.run.smithi177.stderr:15 Sep 03:33:56 ntpd[13744]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-15T03:33:56.414 INFO:teuthology.orchestra.run.smithi177.stderr:15 Sep 03:33:56 ntpd[13744]: can't open /var/log/ntpstats/loopstats.20240915: Permission denied 2024-09-15T03:33:56.482 INFO:teuthology.orchestra.run.smithi177.stdout: remote refid st t when poll reach delay offset jitter 2024-09-15T03:33:56.482 INFO:teuthology.orchestra.run.smithi177.stdout:============================================================================== 2024-09-15T03:33:56.483 INFO:teuthology.orchestra.run.smithi177.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-15T03:33:56.483 INFO:teuthology.orchestra.run.smithi177.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-15T03:33:56.483 INFO:teuthology.orchestra.run.smithi177.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-15T03:33:56.483 INFO:teuthology.orchestra.run.smithi177.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-15T03:33:56.484 INFO:teuthology.run_tasks:Running task install... 2024-09-15T03:33:56.493 DEBUG:teuthology.task.install:project ceph 2024-09-15T03:33:56.493 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}} 2024-09-15T03:33:56.493 DEBUG:teuthology.task.install:config {'extra_system_packages': {'deb': ['sqlite3'], 'rpm': ['sqlite-devel']}, 'flavor': 'default', 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'} 2024-09-15T03:33:56.493 INFO:teuthology.task.install:Using flavor: default 2024-09-15T03:33:56.498 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-15T03:33:56.499 INFO:teuthology.task.install:extra packages: [] 2024-09-15T03:33:56.499 DEBUG:teuthology.orchestra.run.smithi110:> sudo apt-key list | grep Ceph 2024-09-15T03:33:56.500 DEBUG:teuthology.orchestra.run.smithi177:> sudo apt-key list | grep Ceph 2024-09-15T03:33:56.591 INFO:teuthology.orchestra.run.smithi177.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2024-09-15T03:33:56.605 INFO:teuthology.orchestra.run.smithi110.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2024-09-15T03:33:56.654 INFO:teuthology.orchestra.run.smithi177.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-15T03:33:56.655 INFO:teuthology.orchestra.run.smithi177.stdout:uid [ unknown] Ceph.com (release key) 2024-09-15T03:33:56.656 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-15T03:33:56.656 INFO:teuthology.task.install.deb:Installing system (non-project) packages: sqlite3 on remote deb x86_64 2024-09-15T03:33:56.656 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=d75f23820806da622f561ee5ee7551015285993f 2024-09-15T03:33:56.665 INFO:teuthology.orchestra.run.smithi110.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-15T03:33:56.665 INFO:teuthology.orchestra.run.smithi110.stdout:uid [ unknown] Ceph.com (release key) 2024-09-15T03:33:56.666 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-15T03:33:56.666 INFO:teuthology.task.install.deb:Installing system (non-project) packages: sqlite3 on remote deb x86_64 2024-09-15T03:33:56.667 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=d75f23820806da622f561ee5ee7551015285993f 2024-09-15T03:33:56.801 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default/ 2024-09-15T03:33:56.801 INFO:teuthology.task.install.deb:Package version is 17.2.7-1643-gd75f2382-1focal 2024-09-15T03:33:56.812 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default/ 2024-09-15T03:33:56.812 INFO:teuthology.task.install.deb:Package version is 17.2.7-1643-gd75f2382-1focal 2024-09-15T03:33:56.911 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-15T03:33:56.911 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-15T03:33:56.920 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-15T03:33:56.920 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-15T03:33:56.926 DEBUG:teuthology.orchestra.run.smithi177:> sudo apt-get update 2024-09-15T03:33:56.938 DEBUG:teuthology.orchestra.run.smithi110:> sudo apt-get update 2024-09-15T03:33:57.110 INFO:teuthology.orchestra.run.smithi177.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-15T03:33:57.116 INFO:teuthology.orchestra.run.smithi177.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-15T03:33:57.127 INFO:teuthology.orchestra.run.smithi110.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-15T03:33:57.127 INFO:teuthology.orchestra.run.smithi110.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-15T03:33:57.132 INFO:teuthology.orchestra.run.smithi110.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-15T03:33:57.134 INFO:teuthology.orchestra.run.smithi177.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-15T03:33:57.150 INFO:teuthology.orchestra.run.smithi110.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-15T03:33:57.164 INFO:teuthology.orchestra.run.smithi177.stdout:Ign:4 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal InRelease 2024-09-15T03:33:57.177 INFO:teuthology.orchestra.run.smithi110.stdout:Ign:5 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal InRelease 2024-09-15T03:33:57.185 INFO:teuthology.orchestra.run.smithi177.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal Release [7,662 B] 2024-09-15T03:33:57.198 INFO:teuthology.orchestra.run.smithi110.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal Release [7,662 B] 2024-09-15T03:33:57.209 INFO:teuthology.orchestra.run.smithi177.stdout:Ign:6 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal Release.gpg 2024-09-15T03:33:57.221 INFO:teuthology.orchestra.run.smithi110.stdout:Ign:7 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal Release.gpg 2024-09-15T03:33:57.230 INFO:teuthology.orchestra.run.smithi177.stdout:Hit:7 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-15T03:33:57.230 INFO:teuthology.orchestra.run.smithi177.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main i386 Packages [3,392 B] 2024-09-15T03:33:57.243 INFO:teuthology.orchestra.run.smithi110.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main i386 Packages [3,392 B] 2024-09-15T03:33:57.263 INFO:teuthology.orchestra.run.smithi177.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 Packages [17.4 kB] 2024-09-15T03:33:57.275 INFO:teuthology.orchestra.run.smithi110.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 Packages [17.4 kB] 2024-09-15T03:33:57.767 INFO:teuthology.orchestra.run.smithi177.stdout:Fetched 28.5 kB in 1s (44.6 kB/s) 2024-09-15T03:33:57.776 INFO:teuthology.orchestra.run.smithi110.stdout:Fetched 28.5 kB in 1s (45.1 kB/s) 2024-09-15T03:33:58.926 INFO:teuthology.orchestra.run.smithi177.stdout:Reading package lists... 2024-09-15T03:33:58.953 DEBUG:teuthology.orchestra.run.smithi177:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=17.2.7-1643-gd75f2382-1focal cephadm=17.2.7-1643-gd75f2382-1focal ceph-mds=17.2.7-1643-gd75f2382-1focal ceph-mgr=17.2.7-1643-gd75f2382-1focal ceph-common=17.2.7-1643-gd75f2382-1focal ceph-fuse=17.2.7-1643-gd75f2382-1focal ceph-test=17.2.7-1643-gd75f2382-1focal ceph-volume=17.2.7-1643-gd75f2382-1focal radosgw=17.2.7-1643-gd75f2382-1focal python3-rados=17.2.7-1643-gd75f2382-1focal python3-rgw=17.2.7-1643-gd75f2382-1focal python3-cephfs=17.2.7-1643-gd75f2382-1focal python3-rbd=17.2.7-1643-gd75f2382-1focal libcephfs2=17.2.7-1643-gd75f2382-1focal libcephfs-dev=17.2.7-1643-gd75f2382-1focal librados2=17.2.7-1643-gd75f2382-1focal librbd1=17.2.7-1643-gd75f2382-1focal rbd-fuse=17.2.7-1643-gd75f2382-1focal 2024-09-15T03:33:58.954 INFO:teuthology.orchestra.run.smithi110.stdout:Reading package lists... 2024-09-15T03:33:58.981 DEBUG:teuthology.orchestra.run.smithi110:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=17.2.7-1643-gd75f2382-1focal cephadm=17.2.7-1643-gd75f2382-1focal ceph-mds=17.2.7-1643-gd75f2382-1focal ceph-mgr=17.2.7-1643-gd75f2382-1focal ceph-common=17.2.7-1643-gd75f2382-1focal ceph-fuse=17.2.7-1643-gd75f2382-1focal ceph-test=17.2.7-1643-gd75f2382-1focal ceph-volume=17.2.7-1643-gd75f2382-1focal radosgw=17.2.7-1643-gd75f2382-1focal python3-rados=17.2.7-1643-gd75f2382-1focal python3-rgw=17.2.7-1643-gd75f2382-1focal python3-cephfs=17.2.7-1643-gd75f2382-1focal python3-rbd=17.2.7-1643-gd75f2382-1focal libcephfs2=17.2.7-1643-gd75f2382-1focal libcephfs-dev=17.2.7-1643-gd75f2382-1focal librados2=17.2.7-1643-gd75f2382-1focal librbd1=17.2.7-1643-gd75f2382-1focal rbd-fuse=17.2.7-1643-gd75f2382-1focal 2024-09-15T03:33:59.025 INFO:teuthology.orchestra.run.smithi177.stdout:Reading package lists... 2024-09-15T03:33:59.054 INFO:teuthology.orchestra.run.smithi110.stdout:Reading package lists... 2024-09-15T03:33:59.242 INFO:teuthology.orchestra.run.smithi177.stdout:Building dependency tree... 2024-09-15T03:33:59.242 INFO:teuthology.orchestra.run.smithi177.stdout:Reading state information... 2024-09-15T03:33:59.271 INFO:teuthology.orchestra.run.smithi110.stdout:Building dependency tree... 2024-09-15T03:33:59.271 INFO:teuthology.orchestra.run.smithi110.stdout:Reading state information... 2024-09-15T03:33:59.440 INFO:teuthology.orchestra.run.smithi177.stdout:The following packages were automatically installed and are no longer required: 2024-09-15T03:33:59.441 INFO:teuthology.orchestra.run.smithi177.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-09-15T03:33:59.441 INFO:teuthology.orchestra.run.smithi177.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-15T03:33:59.441 INFO:teuthology.orchestra.run.smithi177.stdout:The following additional packages will be installed: 2024-09-15T03:33:59.441 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-15T03:33:59.441 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2024-09-15T03:33:59.441 INFO:teuthology.orchestra.run.smithi177.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2024-09-15T03:33:59.442 INFO:teuthology.orchestra.run.smithi177.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2024-09-15T03:33:59.442 INFO:teuthology.orchestra.run.smithi177.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2024-09-15T03:33:59.442 INFO:teuthology.orchestra.run.smithi177.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2024-09-15T03:33:59.442 INFO:teuthology.orchestra.run.smithi177.stdout: python-natsort-doc python-pastedeploy-tpl python3-atomicwrites python3-attr 2024-09-15T03:33:59.442 INFO:teuthology.orchestra.run.smithi177.stdout: python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2024-09-15T03:33:59.442 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ceph-argparse python3-ceph-common python3-cherrypy3 2024-09-15T03:33:59.443 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cryptography python3-decorator python3-google-auth python3-html5lib 2024-09-15T03:33:59.443 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jinja2 python3-joblib python3-jwt python3-kubernetes 2024-09-15T03:33:59.443 INFO:teuthology.orchestra.run.smithi177.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2024-09-15T03:33:59.443 INFO:teuthology.orchestra.run.smithi177.stdout: python3-natsort python3-oauthlib python3-openssl python3-packaging 2024-09-15T03:33:59.443 INFO:teuthology.orchestra.run.smithi177.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-15T03:33:59.443 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2024-09-15T03:33:59.443 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyasn1-modules python3-pyinotify python3-pytest python3-repoze.lru 2024-09-15T03:33:59.443 INFO:teuthology.orchestra.run.smithi177.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-scipy 2024-09-15T03:33:59.443 INFO:teuthology.orchestra.run.smithi177.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-15T03:33:59.443 INFO:teuthology.orchestra.run.smithi177.stdout: python3-sklearn python3-sklearn-lib python3-soupsieve python3-tempita 2024-09-15T03:33:59.443 INFO:teuthology.orchestra.run.smithi177.stdout: python3-waitress python3-wcwidth python3-webencodings python3-webob 2024-09-15T03:33:59.443 INFO:teuthology.orchestra.run.smithi177.stdout: python3-websocket python3-webtest python3-werkzeug qttranslations5-l10n 2024-09-15T03:33:59.443 INFO:teuthology.orchestra.run.smithi177.stdout: socat xmlstarlet 2024-09-15T03:33:59.444 INFO:teuthology.orchestra.run.smithi177.stdout:Suggested packages: 2024-09-15T03:33:59.444 INFO:teuthology.orchestra.run.smithi177.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2024-09-15T03:33:59.444 INFO:teuthology.orchestra.run.smithi177.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2024-09-15T03:33:59.444 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2024-09-15T03:33:59.444 INFO:teuthology.orchestra.run.smithi177.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2024-09-15T03:33:59.444 INFO:teuthology.orchestra.run.smithi177.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2024-09-15T03:33:59.444 INFO:teuthology.orchestra.run.smithi177.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2024-09-15T03:33:59.444 INFO:teuthology.orchestra.run.smithi177.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2024-09-15T03:33:59.444 INFO:teuthology.orchestra.run.smithi177.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-15T03:33:59.445 INFO:teuthology.orchestra.run.smithi177.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2024-09-15T03:33:59.445 INFO:teuthology.orchestra.run.smithi177.stdout:Recommended packages: 2024-09-15T03:33:59.445 INFO:teuthology.orchestra.run.smithi177.stdout: btrfs-tools 2024-09-15T03:33:59.474 INFO:teuthology.orchestra.run.smithi110.stdout:The following packages were automatically installed and are no longer required: 2024-09-15T03:33:59.475 INFO:teuthology.orchestra.run.smithi110.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-09-15T03:33:59.475 INFO:teuthology.orchestra.run.smithi110.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-15T03:33:59.475 INFO:teuthology.orchestra.run.smithi110.stdout:The following additional packages will be installed: 2024-09-15T03:33:59.475 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-15T03:33:59.475 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2024-09-15T03:33:59.475 INFO:teuthology.orchestra.run.smithi110.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2024-09-15T03:33:59.476 INFO:teuthology.orchestra.run.smithi110.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2024-09-15T03:33:59.476 INFO:teuthology.orchestra.run.smithi110.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2024-09-15T03:33:59.477 INFO:teuthology.orchestra.run.smithi110.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2024-09-15T03:33:59.477 INFO:teuthology.orchestra.run.smithi110.stdout: python-natsort-doc python-pastedeploy-tpl python3-atomicwrites python3-attr 2024-09-15T03:33:59.477 INFO:teuthology.orchestra.run.smithi110.stdout: python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2024-09-15T03:33:59.477 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ceph-argparse python3-ceph-common python3-cherrypy3 2024-09-15T03:33:59.477 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cryptography python3-decorator python3-google-auth python3-html5lib 2024-09-15T03:33:59.477 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jinja2 python3-joblib python3-jwt python3-kubernetes 2024-09-15T03:33:59.477 INFO:teuthology.orchestra.run.smithi110.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2024-09-15T03:33:59.478 INFO:teuthology.orchestra.run.smithi110.stdout: python3-natsort python3-oauthlib python3-openssl python3-packaging 2024-09-15T03:33:59.478 INFO:teuthology.orchestra.run.smithi110.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-15T03:33:59.478 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2024-09-15T03:33:59.478 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyasn1-modules python3-pyinotify python3-pytest python3-repoze.lru 2024-09-15T03:33:59.478 INFO:teuthology.orchestra.run.smithi110.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-scipy 2024-09-15T03:33:59.478 INFO:teuthology.orchestra.run.smithi110.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-15T03:33:59.478 INFO:teuthology.orchestra.run.smithi110.stdout: python3-sklearn python3-sklearn-lib python3-soupsieve python3-tempita 2024-09-15T03:33:59.478 INFO:teuthology.orchestra.run.smithi110.stdout: python3-waitress python3-wcwidth python3-webencodings python3-webob 2024-09-15T03:33:59.478 INFO:teuthology.orchestra.run.smithi110.stdout: python3-websocket python3-webtest python3-werkzeug qttranslations5-l10n 2024-09-15T03:33:59.478 INFO:teuthology.orchestra.run.smithi110.stdout: socat xmlstarlet 2024-09-15T03:33:59.479 INFO:teuthology.orchestra.run.smithi110.stdout:Suggested packages: 2024-09-15T03:33:59.479 INFO:teuthology.orchestra.run.smithi110.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2024-09-15T03:33:59.479 INFO:teuthology.orchestra.run.smithi110.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2024-09-15T03:33:59.479 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2024-09-15T03:33:59.479 INFO:teuthology.orchestra.run.smithi110.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2024-09-15T03:33:59.479 INFO:teuthology.orchestra.run.smithi110.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2024-09-15T03:33:59.479 INFO:teuthology.orchestra.run.smithi110.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2024-09-15T03:33:59.479 INFO:teuthology.orchestra.run.smithi110.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2024-09-15T03:33:59.479 INFO:teuthology.orchestra.run.smithi110.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-15T03:33:59.479 INFO:teuthology.orchestra.run.smithi110.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2024-09-15T03:33:59.480 INFO:teuthology.orchestra.run.smithi110.stdout:Recommended packages: 2024-09-15T03:33:59.480 INFO:teuthology.orchestra.run.smithi110.stdout: btrfs-tools 2024-09-15T03:33:59.564 INFO:teuthology.orchestra.run.smithi177.stdout:The following NEW packages will be installed: 2024-09-15T03:33:59.565 INFO:teuthology.orchestra.run.smithi177.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-15T03:33:59.565 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-15T03:33:59.565 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-15T03:33:59.566 INFO:teuthology.orchestra.run.smithi177.stdout: fonts-mathjax jq libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 2024-09-15T03:33:59.566 INFO:teuthology.orchestra.run.smithi177.stdout: libjs-mathjax libjs-sphinxdoc libjs-underscore liblbfgsb0 liblttng-ust0 2024-09-15T03:33:59.566 INFO:teuthology.orchestra.run.smithi177.stdout: liblua5.3-0 liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-09-15T03:33:59.566 INFO:teuthology.orchestra.run.smithi177.stdout: libqt5network5 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2024-09-15T03:33:59.567 INFO:teuthology.orchestra.run.smithi177.stdout: libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli python-natsort-doc 2024-09-15T03:33:59.567 INFO:teuthology.orchestra.run.smithi177.stdout: python-pastedeploy-tpl python3-atomicwrites python3-attr python3-bcrypt 2024-09-15T03:33:59.567 INFO:teuthology.orchestra.run.smithi177.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2024-09-15T03:33:59.567 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ceph-common python3-cephfs python3-cherrypy3 python3-cryptography 2024-09-15T03:33:59.567 INFO:teuthology.orchestra.run.smithi177.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2024-09-15T03:33:59.567 INFO:teuthology.orchestra.run.smithi177.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2024-09-15T03:33:59.567 INFO:teuthology.orchestra.run.smithi177.stdout: python3-mako python3-markupsafe python3-natsort python3-oauthlib 2024-09-15T03:33:59.567 INFO:teuthology.orchestra.run.smithi177.stdout: python3-openssl python3-packaging python3-paste python3-pastedeploy 2024-09-15T03:33:59.567 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pastescript python3-pecan python3-pluggy python3-prettytable 2024-09-15T03:33:59.567 INFO:teuthology.orchestra.run.smithi177.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-15T03:33:59.568 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-15T03:33:59.568 INFO:teuthology.orchestra.run.smithi177.stdout: python3-repoze.lru python3-requests-oauthlib python3-rgw python3-routes 2024-09-15T03:33:59.568 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2024-09-15T03:33:59.568 INFO:teuthology.orchestra.run.smithi177.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-09-15T03:33:59.568 INFO:teuthology.orchestra.run.smithi177.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-09-15T03:33:59.568 INFO:teuthology.orchestra.run.smithi177.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-09-15T03:33:59.568 INFO:teuthology.orchestra.run.smithi177.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2024-09-15T03:33:59.568 INFO:teuthology.orchestra.run.smithi177.stdout:The following packages will be upgraded: 2024-09-15T03:33:59.569 INFO:teuthology.orchestra.run.smithi177.stdout: librados2 librbd1 2024-09-15T03:33:59.596 INFO:teuthology.orchestra.run.smithi110.stdout:The following NEW packages will be installed: 2024-09-15T03:33:59.597 INFO:teuthology.orchestra.run.smithi110.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-15T03:33:59.597 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-15T03:33:59.597 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-15T03:33:59.597 INFO:teuthology.orchestra.run.smithi110.stdout: fonts-mathjax jq libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 2024-09-15T03:33:59.597 INFO:teuthology.orchestra.run.smithi110.stdout: libjs-mathjax libjs-sphinxdoc libjs-underscore liblbfgsb0 liblttng-ust0 2024-09-15T03:33:59.597 INFO:teuthology.orchestra.run.smithi110.stdout: liblua5.3-0 liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-09-15T03:33:59.597 INFO:teuthology.orchestra.run.smithi110.stdout: libqt5network5 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2024-09-15T03:33:59.599 INFO:teuthology.orchestra.run.smithi110.stdout: libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli python-natsort-doc 2024-09-15T03:33:59.599 INFO:teuthology.orchestra.run.smithi110.stdout: python-pastedeploy-tpl python3-atomicwrites python3-attr python3-bcrypt 2024-09-15T03:33:59.599 INFO:teuthology.orchestra.run.smithi110.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2024-09-15T03:33:59.599 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ceph-common python3-cephfs python3-cherrypy3 python3-cryptography 2024-09-15T03:33:59.599 INFO:teuthology.orchestra.run.smithi110.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2024-09-15T03:33:59.599 INFO:teuthology.orchestra.run.smithi110.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2024-09-15T03:33:59.599 INFO:teuthology.orchestra.run.smithi110.stdout: python3-mako python3-markupsafe python3-natsort python3-oauthlib 2024-09-15T03:33:59.599 INFO:teuthology.orchestra.run.smithi110.stdout: python3-openssl python3-packaging python3-paste python3-pastedeploy 2024-09-15T03:33:59.600 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pastescript python3-pecan python3-pluggy python3-prettytable 2024-09-15T03:33:59.600 INFO:teuthology.orchestra.run.smithi110.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-15T03:33:59.600 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-15T03:33:59.600 INFO:teuthology.orchestra.run.smithi110.stdout: python3-repoze.lru python3-requests-oauthlib python3-rgw python3-routes 2024-09-15T03:33:59.600 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2024-09-15T03:33:59.600 INFO:teuthology.orchestra.run.smithi110.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-09-15T03:33:59.600 INFO:teuthology.orchestra.run.smithi110.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-09-15T03:33:59.600 INFO:teuthology.orchestra.run.smithi110.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-09-15T03:33:59.600 INFO:teuthology.orchestra.run.smithi110.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2024-09-15T03:33:59.600 INFO:teuthology.orchestra.run.smithi110.stdout:The following packages will be upgraded: 2024-09-15T03:33:59.601 INFO:teuthology.orchestra.run.smithi110.stdout: librados2 librbd1 2024-09-15T03:33:59.632 INFO:teuthology.orchestra.run.smithi177.stdout:2 upgraded, 108 newly installed, 0 to remove and 295 not upgraded. 2024-09-15T03:33:59.632 INFO:teuthology.orchestra.run.smithi177.stdout:Need to get 201 MB of archives. 2024-09-15T03:33:59.632 INFO:teuthology.orchestra.run.smithi177.stdout:After this operation, 938 MB of additional disk space will be used. 2024-09-15T03:33:59.633 INFO:teuthology.orchestra.run.smithi177.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2024-09-15T03:33:59.663 INFO:teuthology.orchestra.run.smithi110.stdout:2 upgraded, 108 newly installed, 0 to remove and 295 not upgraded. 2024-09-15T03:33:59.663 INFO:teuthology.orchestra.run.smithi110.stdout:Need to get 201 MB of archives. 2024-09-15T03:33:59.663 INFO:teuthology.orchestra.run.smithi110.stdout:After this operation, 938 MB of additional disk space will be used. 2024-09-15T03:33:59.663 INFO:teuthology.orchestra.run.smithi110.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2024-09-15T03:33:59.716 INFO:teuthology.orchestra.run.smithi177.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2024-09-15T03:33:59.718 INFO:teuthology.orchestra.run.smithi177.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2024-09-15T03:33:59.718 INFO:teuthology.orchestra.run.smithi177.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 17.2.7-1643-gd75f2382-1focal [3,508 kB] 2024-09-15T03:33:59.727 INFO:teuthology.orchestra.run.smithi177.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2024-09-15T03:33:59.733 INFO:teuthology.orchestra.run.smithi177.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2024-09-15T03:33:59.749 INFO:teuthology.orchestra.run.smithi110.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2024-09-15T03:33:59.750 INFO:teuthology.orchestra.run.smithi110.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2024-09-15T03:33:59.751 INFO:teuthology.orchestra.run.smithi110.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 17.2.7-1643-gd75f2382-1focal [3,508 kB] 2024-09-15T03:33:59.758 INFO:teuthology.orchestra.run.smithi177.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2024-09-15T03:33:59.759 INFO:teuthology.orchestra.run.smithi177.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2024-09-15T03:33:59.759 INFO:teuthology.orchestra.run.smithi110.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2024-09-15T03:33:59.765 INFO:teuthology.orchestra.run.smithi177.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2024-09-15T03:33:59.765 INFO:teuthology.orchestra.run.smithi110.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2024-09-15T03:33:59.789 INFO:teuthology.orchestra.run.smithi110.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2024-09-15T03:33:59.791 INFO:teuthology.orchestra.run.smithi110.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2024-09-15T03:33:59.796 INFO:teuthology.orchestra.run.smithi110.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2024-09-15T03:33:59.797 INFO:teuthology.orchestra.run.smithi177.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2024-09-15T03:33:59.798 INFO:teuthology.orchestra.run.smithi177.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2024-09-15T03:33:59.805 INFO:teuthology.orchestra.run.smithi177.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2024-09-15T03:33:59.806 INFO:teuthology.orchestra.run.smithi177.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2024-09-15T03:33:59.806 INFO:teuthology.orchestra.run.smithi177.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.3 [211 kB] 2024-09-15T03:33:59.808 INFO:teuthology.orchestra.run.smithi177.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2024-09-15T03:33:59.809 INFO:teuthology.orchestra.run.smithi177.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2024-09-15T03:33:59.809 INFO:teuthology.orchestra.run.smithi177.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2024-09-15T03:33:59.809 INFO:teuthology.orchestra.run.smithi177.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2024-09-15T03:33:59.815 INFO:teuthology.orchestra.run.smithi177.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2024-09-15T03:33:59.827 INFO:teuthology.orchestra.run.smithi110.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2024-09-15T03:33:59.829 INFO:teuthology.orchestra.run.smithi177.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2024-09-15T03:33:59.830 INFO:teuthology.orchestra.run.smithi110.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2024-09-15T03:33:59.830 INFO:teuthology.orchestra.run.smithi177.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2024-09-15T03:33:59.831 INFO:teuthology.orchestra.run.smithi177.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2024-09-15T03:33:59.831 INFO:teuthology.orchestra.run.smithi177.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2024-09-15T03:33:59.832 INFO:teuthology.orchestra.run.smithi177.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-15T03:33:59.832 INFO:teuthology.orchestra.run.smithi177.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2024-09-15T03:33:59.832 INFO:teuthology.orchestra.run.smithi177.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-webob all 1:1.8.5-2ubuntu0.1 [84.5 kB] 2024-09-15T03:33:59.833 INFO:teuthology.orchestra.run.smithi177.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2024-09-15T03:33:59.834 INFO:teuthology.orchestra.run.smithi177.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2024-09-15T03:33:59.835 INFO:teuthology.orchestra.run.smithi110.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2024-09-15T03:33:59.837 INFO:teuthology.orchestra.run.smithi110.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2024-09-15T03:33:59.837 INFO:teuthology.orchestra.run.smithi110.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.3 [211 kB] 2024-09-15T03:33:59.839 INFO:teuthology.orchestra.run.smithi110.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2024-09-15T03:33:59.840 INFO:teuthology.orchestra.run.smithi110.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2024-09-15T03:33:59.840 INFO:teuthology.orchestra.run.smithi110.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2024-09-15T03:33:59.841 INFO:teuthology.orchestra.run.smithi110.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2024-09-15T03:33:59.844 INFO:teuthology.orchestra.run.smithi110.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2024-09-15T03:33:59.848 INFO:teuthology.orchestra.run.smithi177.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2024-09-15T03:33:59.849 INFO:teuthology.orchestra.run.smithi177.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2024-09-15T03:33:59.850 INFO:teuthology.orchestra.run.smithi177.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2024-09-15T03:33:59.855 INFO:teuthology.orchestra.run.smithi177.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2024-09-15T03:33:59.859 INFO:teuthology.orchestra.run.smithi110.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2024-09-15T03:33:59.860 INFO:teuthology.orchestra.run.smithi110.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2024-09-15T03:33:59.860 INFO:teuthology.orchestra.run.smithi110.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2024-09-15T03:33:59.860 INFO:teuthology.orchestra.run.smithi110.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2024-09-15T03:33:59.861 INFO:teuthology.orchestra.run.smithi110.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-15T03:33:59.862 INFO:teuthology.orchestra.run.smithi110.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2024-09-15T03:33:59.862 INFO:teuthology.orchestra.run.smithi110.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-webob all 1:1.8.5-2ubuntu0.1 [84.5 kB] 2024-09-15T03:33:59.864 INFO:teuthology.orchestra.run.smithi110.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2024-09-15T03:33:59.865 INFO:teuthology.orchestra.run.smithi110.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2024-09-15T03:33:59.865 INFO:teuthology.orchestra.run.smithi177.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2024-09-15T03:33:59.866 INFO:teuthology.orchestra.run.smithi177.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2024-09-15T03:33:59.867 INFO:teuthology.orchestra.run.smithi177.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2024-09-15T03:33:59.868 INFO:teuthology.orchestra.run.smithi177.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.2 [183 kB] 2024-09-15T03:33:59.872 INFO:teuthology.orchestra.run.smithi177.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jinja2 all 2.10.1-2ubuntu0.3 [96.0 kB] 2024-09-15T03:33:59.874 INFO:teuthology.orchestra.run.smithi177.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-15T03:33:59.878 INFO:teuthology.orchestra.run.smithi110.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2024-09-15T03:33:59.879 INFO:teuthology.orchestra.run.smithi110.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2024-09-15T03:33:59.880 INFO:teuthology.orchestra.run.smithi110.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2024-09-15T03:33:59.882 INFO:teuthology.orchestra.run.smithi177.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2024-09-15T03:33:59.884 INFO:teuthology.orchestra.run.smithi177.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2024-09-15T03:33:59.884 INFO:teuthology.orchestra.run.smithi177.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2024-09-15T03:33:59.885 INFO:teuthology.orchestra.run.smithi177.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2024-09-15T03:33:59.895 INFO:teuthology.orchestra.run.smithi110.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2024-09-15T03:33:59.895 INFO:teuthology.orchestra.run.smithi110.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2024-09-15T03:33:59.895 INFO:teuthology.orchestra.run.smithi110.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2024-09-15T03:33:59.896 INFO:teuthology.orchestra.run.smithi110.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2024-09-15T03:33:59.897 INFO:teuthology.orchestra.run.smithi110.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.2 [183 kB] 2024-09-15T03:33:59.899 INFO:teuthology.orchestra.run.smithi177.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2024-09-15T03:33:59.900 INFO:teuthology.orchestra.run.smithi177.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2024-09-15T03:33:59.900 INFO:teuthology.orchestra.run.smithi110.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jinja2 all 2.10.1-2ubuntu0.3 [96.0 kB] 2024-09-15T03:33:59.902 INFO:teuthology.orchestra.run.smithi110.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-15T03:33:59.911 INFO:teuthology.orchestra.run.smithi110.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2024-09-15T03:33:59.913 INFO:teuthology.orchestra.run.smithi110.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2024-09-15T03:33:59.914 INFO:teuthology.orchestra.run.smithi110.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2024-09-15T03:33:59.920 INFO:teuthology.orchestra.run.smithi177.stdout:Get:45 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 17.2.7-1643-gd75f2382-1focal [3,564 kB] 2024-09-15T03:33:59.928 INFO:teuthology.orchestra.run.smithi110.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2024-09-15T03:33:59.929 INFO:teuthology.orchestra.run.smithi110.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2024-09-15T03:33:59.929 INFO:teuthology.orchestra.run.smithi110.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2024-09-15T03:33:59.982 INFO:teuthology.orchestra.run.smithi110.stdout:Get:45 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 17.2.7-1643-gd75f2382-1focal [3,564 kB] 2024-09-15T03:33:59.983 INFO:teuthology.orchestra.run.smithi177.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2024-09-15T03:34:00.002 INFO:teuthology.orchestra.run.smithi177.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2024-09-15T03:34:00.011 INFO:teuthology.orchestra.run.smithi110.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2024-09-15T03:34:00.079 INFO:teuthology.orchestra.run.smithi177.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2024-09-15T03:34:00.079 INFO:teuthology.orchestra.run.smithi177.stdout:Get:49 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 17.2.7-1643-gd75f2382-1focal [761 kB] 2024-09-15T03:34:00.092 INFO:teuthology.orchestra.run.smithi177.stdout:Get:50 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 17.2.7-1643-gd75f2382-1focal [414 kB] 2024-09-15T03:34:00.095 INFO:teuthology.orchestra.run.smithi177.stdout:Get:51 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 17.2.7-1643-gd75f2382-1focal [22.6 kB] 2024-09-15T03:34:00.095 INFO:teuthology.orchestra.run.smithi177.stdout:Get:52 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 17.2.7-1643-gd75f2382-1focal [200 kB] 2024-09-15T03:34:00.097 INFO:teuthology.orchestra.run.smithi177.stdout:Get:53 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 17.2.7-1643-gd75f2382-1focal [49.1 kB] 2024-09-15T03:34:00.097 INFO:teuthology.orchestra.run.smithi177.stdout:Get:54 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 17.2.7-1643-gd75f2382-1focal [389 kB] 2024-09-15T03:34:00.098 INFO:teuthology.orchestra.run.smithi177.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2024-09-15T03:34:00.099 INFO:teuthology.orchestra.run.smithi177.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2024-09-15T03:34:00.099 INFO:teuthology.orchestra.run.smithi177.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2024-09-15T03:34:00.099 INFO:teuthology.orchestra.run.smithi177.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2024-09-15T03:34:00.100 INFO:teuthology.orchestra.run.smithi177.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2024-09-15T03:34:00.100 INFO:teuthology.orchestra.run.smithi177.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2024-09-15T03:34:00.101 INFO:teuthology.orchestra.run.smithi177.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2024-09-15T03:34:00.101 INFO:teuthology.orchestra.run.smithi177.stdout:Get:62 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 17.2.7-1643-gd75f2382-1focal [5,637 kB] 2024-09-15T03:34:00.101 INFO:teuthology.orchestra.run.smithi177.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2024-09-15T03:34:00.102 INFO:teuthology.orchestra.run.smithi177.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2024-09-15T03:34:00.110 INFO:teuthology.orchestra.run.smithi110.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2024-09-15T03:34:00.111 INFO:teuthology.orchestra.run.smithi110.stdout:Get:48 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 17.2.7-1643-gd75f2382-1focal [761 kB] 2024-09-15T03:34:00.113 INFO:teuthology.orchestra.run.smithi110.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2024-09-15T03:34:00.116 INFO:teuthology.orchestra.run.smithi177.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2024-09-15T03:34:00.119 INFO:teuthology.orchestra.run.smithi177.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2024-09-15T03:34:00.125 INFO:teuthology.orchestra.run.smithi110.stdout:Get:50 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 17.2.7-1643-gd75f2382-1focal [414 kB] 2024-09-15T03:34:00.129 INFO:teuthology.orchestra.run.smithi110.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2024-09-15T03:34:00.129 INFO:teuthology.orchestra.run.smithi110.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2024-09-15T03:34:00.130 INFO:teuthology.orchestra.run.smithi110.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2024-09-15T03:34:00.130 INFO:teuthology.orchestra.run.smithi110.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2024-09-15T03:34:00.131 INFO:teuthology.orchestra.run.smithi110.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2024-09-15T03:34:00.131 INFO:teuthology.orchestra.run.smithi110.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2024-09-15T03:34:00.131 INFO:teuthology.orchestra.run.smithi110.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2024-09-15T03:34:00.132 INFO:teuthology.orchestra.run.smithi110.stdout:Get:58 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 17.2.7-1643-gd75f2382-1focal [22.6 kB] 2024-09-15T03:34:00.133 INFO:teuthology.orchestra.run.smithi110.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2024-09-15T03:34:00.133 INFO:teuthology.orchestra.run.smithi110.stdout:Get:60 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 17.2.7-1643-gd75f2382-1focal [200 kB] 2024-09-15T03:34:00.133 INFO:teuthology.orchestra.run.smithi177.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2024-09-15T03:34:00.135 INFO:teuthology.orchestra.run.smithi177.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2024-09-15T03:34:00.136 INFO:teuthology.orchestra.run.smithi177.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2024-09-15T03:34:00.140 INFO:teuthology.orchestra.run.smithi177.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2024-09-15T03:34:00.143 INFO:teuthology.orchestra.run.smithi177.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2024-09-15T03:34:00.145 INFO:teuthology.orchestra.run.smithi110.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2024-09-15T03:34:00.150 INFO:teuthology.orchestra.run.smithi110.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2024-09-15T03:34:00.153 INFO:teuthology.orchestra.run.smithi110.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2024-09-15T03:34:00.155 INFO:teuthology.orchestra.run.smithi110.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2024-09-15T03:34:00.157 INFO:teuthology.orchestra.run.smithi110.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2024-09-15T03:34:00.158 INFO:teuthology.orchestra.run.smithi110.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2024-09-15T03:34:00.158 INFO:teuthology.orchestra.run.smithi110.stdout:Get:67 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 17.2.7-1643-gd75f2382-1focal [49.1 kB] 2024-09-15T03:34:00.159 INFO:teuthology.orchestra.run.smithi110.stdout:Get:68 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 17.2.7-1643-gd75f2382-1focal [389 kB] 2024-09-15T03:34:00.160 INFO:teuthology.orchestra.run.smithi177.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.2 [283 kB] 2024-09-15T03:34:00.162 INFO:teuthology.orchestra.run.smithi177.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2024-09-15T03:34:00.162 INFO:teuthology.orchestra.run.smithi110.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2024-09-15T03:34:00.164 INFO:teuthology.orchestra.run.smithi110.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2024-09-15T03:34:00.175 INFO:teuthology.orchestra.run.smithi110.stdout:Get:71 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 17.2.7-1643-gd75f2382-1focal [5,637 kB] 2024-09-15T03:34:00.179 INFO:teuthology.orchestra.run.smithi110.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.2 [283 kB] 2024-09-15T03:34:00.181 INFO:teuthology.orchestra.run.smithi110.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2024-09-15T03:34:00.204 INFO:teuthology.orchestra.run.smithi177.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2024-09-15T03:34:00.205 INFO:teuthology.orchestra.run.smithi177.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2024-09-15T03:34:00.205 INFO:teuthology.orchestra.run.smithi177.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2024-09-15T03:34:00.206 INFO:teuthology.orchestra.run.smithi177.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2024-09-15T03:34:00.206 INFO:teuthology.orchestra.run.smithi177.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2024-09-15T03:34:00.207 INFO:teuthology.orchestra.run.smithi177.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2024-09-15T03:34:00.220 INFO:teuthology.orchestra.run.smithi110.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2024-09-15T03:34:00.221 INFO:teuthology.orchestra.run.smithi110.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2024-09-15T03:34:00.222 INFO:teuthology.orchestra.run.smithi110.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2024-09-15T03:34:00.222 INFO:teuthology.orchestra.run.smithi110.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2024-09-15T03:34:00.222 INFO:teuthology.orchestra.run.smithi110.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2024-09-15T03:34:00.223 INFO:teuthology.orchestra.run.smithi110.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2024-09-15T03:34:00.235 INFO:teuthology.orchestra.run.smithi110.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-15T03:34:00.235 INFO:teuthology.orchestra.run.smithi110.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2024-09-15T03:34:00.235 INFO:teuthology.orchestra.run.smithi110.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2024-09-15T03:34:00.238 INFO:teuthology.orchestra.run.smithi110.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2024-09-15T03:34:00.255 INFO:teuthology.orchestra.run.smithi110.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2024-09-15T03:34:00.255 INFO:teuthology.orchestra.run.smithi110.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2024-09-15T03:34:00.256 INFO:teuthology.orchestra.run.smithi110.stdout:Get:86 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2024-09-15T03:34:00.258 INFO:teuthology.orchestra.run.smithi110.stdout:Get:87 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2024-09-15T03:34:00.259 INFO:teuthology.orchestra.run.smithi110.stdout:Get:88 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2024-09-15T03:34:00.259 INFO:teuthology.orchestra.run.smithi177.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-15T03:34:00.259 INFO:teuthology.orchestra.run.smithi177.stdout:Get:81 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 17.2.7-1643-gd75f2382-1focal [116 kB] 2024-09-15T03:34:00.260 INFO:teuthology.orchestra.run.smithi177.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2024-09-15T03:34:00.260 INFO:teuthology.orchestra.run.smithi177.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2024-09-15T03:34:00.261 INFO:teuthology.orchestra.run.smithi177.stdout:Get:84 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 17.2.7-1643-gd75f2382-1focal [479 kB] 2024-09-15T03:34:00.262 INFO:teuthology.orchestra.run.smithi177.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2024-09-15T03:34:00.262 INFO:teuthology.orchestra.run.smithi177.stdout:Get:86 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2024-09-15T03:34:00.263 INFO:teuthology.orchestra.run.smithi177.stdout:Get:87 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2024-09-15T03:34:00.291 INFO:teuthology.orchestra.run.smithi177.stdout:Get:88 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2024-09-15T03:34:00.291 INFO:teuthology.orchestra.run.smithi177.stdout:Get:89 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 17.2.7-1643-gd75f2382-1focal [27.0 MB] 2024-09-15T03:34:00.292 INFO:teuthology.orchestra.run.smithi177.stdout:Get:90 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2024-09-15T03:34:00.293 INFO:teuthology.orchestra.run.smithi177.stdout:Get:91 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2024-09-15T03:34:00.393 INFO:teuthology.orchestra.run.smithi110.stdout:Get:89 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 17.2.7-1643-gd75f2382-1focal [116 kB] 2024-09-15T03:34:00.396 INFO:teuthology.orchestra.run.smithi110.stdout:Get:90 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 17.2.7-1643-gd75f2382-1focal [479 kB] 2024-09-15T03:34:00.401 INFO:teuthology.orchestra.run.smithi110.stdout:Get:91 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 17.2.7-1643-gd75f2382-1focal [27.0 MB] 2024-09-15T03:34:01.105 INFO:teuthology.orchestra.run.smithi177.stdout:Get:92 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 17.2.7-1643-gd75f2382-1focal [5,178 kB] 2024-09-15T03:34:01.197 INFO:teuthology.orchestra.run.smithi177.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 17.2.7-1643-gd75f2382-1focal [223 kB] 2024-09-15T03:34:01.200 INFO:teuthology.orchestra.run.smithi177.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 17.2.7-1643-gd75f2382-1focal [157 kB] 2024-09-15T03:34:01.203 INFO:teuthology.orchestra.run.smithi177.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 17.2.7-1643-gd75f2382-1focal [1,594 kB] 2024-09-15T03:34:01.254 INFO:teuthology.orchestra.run.smithi177.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 17.2.7-1643-gd75f2382-1focal [4,684 kB] 2024-09-15T03:34:01.407 INFO:teuthology.orchestra.run.smithi177.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 17.2.7-1643-gd75f2382-1focal [23.1 MB] 2024-09-15T03:34:01.520 INFO:teuthology.orchestra.run.smithi110.stdout:Get:92 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 17.2.7-1643-gd75f2382-1focal [5,178 kB] 2024-09-15T03:34:01.600 INFO:teuthology.orchestra.run.smithi110.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 17.2.7-1643-gd75f2382-1focal [223 kB] 2024-09-15T03:34:01.616 INFO:teuthology.orchestra.run.smithi110.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 17.2.7-1643-gd75f2382-1focal [157 kB] 2024-09-15T03:34:01.620 INFO:teuthology.orchestra.run.smithi110.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 17.2.7-1643-gd75f2382-1focal [1,594 kB] 2024-09-15T03:34:01.676 INFO:teuthology.orchestra.run.smithi110.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 17.2.7-1643-gd75f2382-1focal [4,684 kB] 2024-09-15T03:34:01.841 INFO:teuthology.orchestra.run.smithi110.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 17.2.7-1643-gd75f2382-1focal [23.1 MB] 2024-09-15T03:34:02.173 INFO:teuthology.orchestra.run.smithi177.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 17.2.7-1643-gd75f2382-1focal [3,940 B] 2024-09-15T03:34:02.173 INFO:teuthology.orchestra.run.smithi177.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 17.2.7-1643-gd75f2382-1focal [871 kB] 2024-09-15T03:34:02.185 INFO:teuthology.orchestra.run.smithi177.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 17.2.7-1643-gd75f2382-1focal [2,418 kB] 2024-09-15T03:34:02.206 INFO:teuthology.orchestra.run.smithi177.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 17.2.7-1643-gd75f2382-1focal [81.4 kB] 2024-09-15T03:34:02.207 INFO:teuthology.orchestra.run.smithi177.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 17.2.7-1643-gd75f2382-1focal [120 kB] 2024-09-15T03:34:02.208 INFO:teuthology.orchestra.run.smithi177.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 17.2.7-1643-gd75f2382-1focal [6,653 kB] 2024-09-15T03:34:02.420 INFO:teuthology.orchestra.run.smithi177.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 17.2.7-1643-gd75f2382-1focal [7,885 kB] 2024-09-15T03:34:02.663 INFO:teuthology.orchestra.run.smithi177.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 17.2.7-1643-gd75f2382-1focal [4,064 B] 2024-09-15T03:34:02.663 INFO:teuthology.orchestra.run.smithi177.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 17.2.7-1643-gd75f2382-1focal [53.4 MB] 2024-09-15T03:34:02.726 INFO:teuthology.orchestra.run.smithi110.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 17.2.7-1643-gd75f2382-1focal [3,940 B] 2024-09-15T03:34:02.726 INFO:teuthology.orchestra.run.smithi110.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 17.2.7-1643-gd75f2382-1focal [871 kB] 2024-09-15T03:34:02.740 INFO:teuthology.orchestra.run.smithi110.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 17.2.7-1643-gd75f2382-1focal [2,418 kB] 2024-09-15T03:34:02.758 INFO:teuthology.orchestra.run.smithi110.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 17.2.7-1643-gd75f2382-1focal [81.4 kB] 2024-09-15T03:34:02.758 INFO:teuthology.orchestra.run.smithi110.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 17.2.7-1643-gd75f2382-1focal [120 kB] 2024-09-15T03:34:02.759 INFO:teuthology.orchestra.run.smithi110.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 17.2.7-1643-gd75f2382-1focal [6,653 kB] 2024-09-15T03:34:03.008 INFO:teuthology.orchestra.run.smithi110.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 17.2.7-1643-gd75f2382-1focal [7,885 kB] 2024-09-15T03:34:03.316 INFO:teuthology.orchestra.run.smithi110.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 17.2.7-1643-gd75f2382-1focal [4,064 B] 2024-09-15T03:34:03.317 INFO:teuthology.orchestra.run.smithi110.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 17.2.7-1643-gd75f2382-1focal [53.4 MB] 2024-09-15T03:34:04.476 INFO:teuthology.orchestra.run.smithi177.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 17.2.7-1643-gd75f2382-1focal [231 kB] 2024-09-15T03:34:04.482 INFO:teuthology.orchestra.run.smithi177.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 17.2.7-1643-gd75f2382-1focal [21.9 kB] 2024-09-15T03:34:04.482 INFO:teuthology.orchestra.run.smithi177.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 17.2.7-1643-gd75f2382-1focal [17.2 MB] 2024-09-15T03:34:04.998 INFO:teuthology.orchestra.run.smithi177.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 17.2.7-1643-gd75f2382-1focal [80.7 kB] 2024-09-15T03:34:05.233 INFO:teuthology.orchestra.run.smithi110.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 17.2.7-1643-gd75f2382-1focal [231 kB] 2024-09-15T03:34:05.237 INFO:teuthology.orchestra.run.smithi110.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 17.2.7-1643-gd75f2382-1focal [21.9 kB] 2024-09-15T03:34:05.249 INFO:teuthology.orchestra.run.smithi110.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 17.2.7-1643-gd75f2382-1focal [17.2 MB] 2024-09-15T03:34:05.546 INFO:teuthology.orchestra.run.smithi110.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 17.2.7-1643-gd75f2382-1focal [80.7 kB] 2024-09-15T03:34:05.643 INFO:teuthology.orchestra.run.smithi177.stdout:Fetched 201 MB in 5s (37.1 MB/s) 2024-09-15T03:34:05.800 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2024-09-15T03:34:05.859 INFO:teuthology.orchestra.run.smithi177.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141720 files and directories currently installed.) 2024-09-15T03:34:05.864 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../000-liblttng-ust0_2.11.0-1_amd64.deb ... 2024-09-15T03:34:05.965 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-15T03:34:06.175 INFO:teuthology.orchestra.run.smithi110.stdout:Fetched 201 MB in 6s (33.9 MB/s) 2024-09-15T03:34:06.305 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../001-librbd1_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:34:06.349 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2024-09-15T03:34:06.405 INFO:teuthology.orchestra.run.smithi110.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141720 files and directories currently installed.) 2024-09-15T03:34:06.409 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../000-liblttng-ust0_2.11.0-1_amd64.deb ... 2024-09-15T03:34:06.410 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking librbd1 (17.2.7-1643-gd75f2382-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-15T03:34:06.489 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-15T03:34:06.819 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../001-librbd1_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:34:06.952 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking librbd1 (17.2.7-1643-gd75f2382-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-15T03:34:07.117 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../002-librados2_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:34:07.239 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking librados2 (17.2.7-1643-gd75f2382-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-15T03:34:07.624 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../002-librados2_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:34:07.731 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking librados2 (17.2.7-1643-gd75f2382-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-15T03:34:07.935 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package libcephfs2. 2024-09-15T03:34:07.951 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../003-libcephfs2_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:34:07.993 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking libcephfs2 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:34:08.304 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package python3-rados. 2024-09-15T03:34:08.320 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../004-python3-rados_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:34:08.354 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking python3-rados (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:34:08.368 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package libcephfs2. 2024-09-15T03:34:08.385 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../003-libcephfs2_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:34:08.427 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking libcephfs2 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:34:08.690 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-15T03:34:08.706 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../005-python3-ceph-argparse_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-15T03:34:08.740 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking python3-ceph-argparse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:34:08.771 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-rados. 2024-09-15T03:34:08.788 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../004-python3-rados_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:34:08.829 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-rados (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:34:09.000 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package python3-cephfs. 2024-09-15T03:34:09.020 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../006-python3-cephfs_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:34:09.058 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking python3-cephfs (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:34:09.157 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-15T03:34:09.175 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../005-python3-ceph-argparse_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-15T03:34:09.207 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-ceph-argparse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:34:09.395 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-15T03:34:09.411 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../007-python3-ceph-common_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-15T03:34:09.434 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-cephfs. 2024-09-15T03:34:09.450 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../006-python3-cephfs_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:34:09.453 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking python3-ceph-common (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:34:09.493 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-cephfs (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:34:09.798 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package python3-prettytable. 2024-09-15T03:34:09.815 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../008-python3-prettytable_0.7.2-5_all.deb ... 2024-09-15T03:34:09.837 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-15T03:34:09.854 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../007-python3-ceph-common_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-15T03:34:09.856 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2024-09-15T03:34:09.921 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-ceph-common (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:34:10.109 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package python3-rbd. 2024-09-15T03:34:10.124 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../009-python3-rbd_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:34:10.167 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking python3-rbd (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:34:10.241 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-prettytable. 2024-09-15T03:34:10.256 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../008-python3-prettytable_0.7.2-5_all.deb ... 2024-09-15T03:34:10.299 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2024-09-15T03:34:10.527 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-rbd. 2024-09-15T03:34:10.536 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2024-09-15T03:34:10.544 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../009-python3-rbd_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:34:10.554 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../010-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2024-09-15T03:34:10.576 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-rbd (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:34:10.594 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-15T03:34:10.954 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2024-09-15T03:34:10.972 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../010-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2024-09-15T03:34:11.001 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package librabbitmq4:amd64. 2024-09-15T03:34:11.013 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-15T03:34:11.017 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../011-librabbitmq4_0.10.0-1_amd64.deb ... 2024-09-15T03:34:11.056 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2024-09-15T03:34:11.366 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package librabbitmq4:amd64. 2024-09-15T03:34:11.384 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../011-librabbitmq4_0.10.0-1_amd64.deb ... 2024-09-15T03:34:11.392 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-15T03:34:11.414 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../012-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2024-09-15T03:34:11.424 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2024-09-15T03:34:11.450 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-15T03:34:11.719 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-15T03:34:11.738 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../012-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2024-09-15T03:34:11.769 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-15T03:34:11.786 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-15T03:34:11.803 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../013-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2024-09-15T03:34:11.836 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-15T03:34:12.131 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-15T03:34:12.151 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../014-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2024-09-15T03:34:12.164 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-15T03:34:12.180 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../013-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2024-09-15T03:34:12.181 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-15T03:34:12.222 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-15T03:34:12.558 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-15T03:34:12.574 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../014-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2024-09-15T03:34:12.616 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-15T03:34:12.684 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-15T03:34:12.700 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../015-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-15T03:34:12.753 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-15T03:34:13.112 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-15T03:34:13.129 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../015-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-15T03:34:13.181 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-15T03:34:13.278 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-15T03:34:13.297 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../016-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-15T03:34:13.336 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-15T03:34:13.674 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-15T03:34:13.681 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-15T03:34:13.690 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../016-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-15T03:34:13.701 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../017-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-15T03:34:13.731 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-15T03:34:13.732 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-15T03:34:14.109 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package libthrift-0.13.0. 2024-09-15T03:34:14.110 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-15T03:34:14.126 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../017-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-15T03:34:14.126 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../018-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2024-09-15T03:34:14.168 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-15T03:34:14.168 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-15T03:34:14.547 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package libthrift-0.13.0. 2024-09-15T03:34:14.553 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package librgw2. 2024-09-15T03:34:14.564 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../018-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2024-09-15T03:34:14.571 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../019-librgw2_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:34:14.605 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-15T03:34:14.612 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking librgw2 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:34:14.983 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package librgw2. 2024-09-15T03:34:15.003 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../019-librgw2_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:34:15.041 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking librgw2 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:34:15.364 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package python3-rgw. 2024-09-15T03:34:15.382 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../020-python3-rgw_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:34:15.423 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking python3-rgw (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:34:15.783 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package liboath0. 2024-09-15T03:34:15.800 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../021-liboath0_2.6.1-1.3_amd64.deb ... 2024-09-15T03:34:15.819 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-rgw. 2024-09-15T03:34:15.838 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../020-python3-rgw_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:34:15.841 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2024-09-15T03:34:15.894 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-rgw (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:34:16.153 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package libradosstriper1. 2024-09-15T03:34:16.168 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../022-libradosstriper1_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:34:16.211 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking libradosstriper1 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:34:16.247 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package liboath0. 2024-09-15T03:34:16.263 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../021-liboath0_2.6.1-1.3_amd64.deb ... 2024-09-15T03:34:16.297 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2024-09-15T03:34:16.775 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package libradosstriper1. 2024-09-15T03:34:16.794 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../022-libradosstriper1_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:34:16.866 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking libradosstriper1 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:34:16.931 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package ceph-common. 2024-09-15T03:34:16.948 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../023-ceph-common_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:34:16.989 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking ceph-common (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:34:17.311 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package ceph-common. 2024-09-15T03:34:17.334 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../023-ceph-common_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:34:17.370 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking ceph-common (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:34:19.469 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package ceph-base. 2024-09-15T03:34:19.485 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../024-ceph-base_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:34:19.538 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking ceph-base (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:34:19.909 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package ceph-base. 2024-09-15T03:34:19.925 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../024-ceph-base_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:34:19.977 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking ceph-base (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:34:20.324 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package python3-cryptography. 2024-09-15T03:34:20.340 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../025-python3-cryptography_2.8-3ubuntu0.3_amd64.deb ... 2024-09-15T03:34:20.417 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-15T03:34:20.752 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package python3-openssl. 2024-09-15T03:34:20.768 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../026-python3-openssl_19.0.0-1build1_all.deb ... 2024-09-15T03:34:20.810 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2024-09-15T03:34:20.880 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-cryptography. 2024-09-15T03:34:20.896 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../025-python3-cryptography_2.8-3ubuntu0.3_amd64.deb ... 2024-09-15T03:34:20.938 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-15T03:34:21.163 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package python3-packaging. 2024-09-15T03:34:21.179 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../027-python3-packaging_20.3-1_all.deb ... 2024-09-15T03:34:21.221 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking python3-packaging (20.3-1) ... 2024-09-15T03:34:21.334 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-openssl. 2024-09-15T03:34:21.354 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../026-python3-openssl_19.0.0-1build1_all.deb ... 2024-09-15T03:34:21.392 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2024-09-15T03:34:21.474 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-15T03:34:21.491 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../028-ceph-mgr-modules-core_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-15T03:34:21.532 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking ceph-mgr-modules-core (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:34:21.720 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-packaging. 2024-09-15T03:34:21.737 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../027-python3-packaging_20.3-1_all.deb ... 2024-09-15T03:34:21.798 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-packaging (20.3-1) ... 2024-09-15T03:34:21.937 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-15T03:34:21.953 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../029-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2024-09-15T03:34:21.995 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-15T03:34:22.056 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-15T03:34:22.074 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../028-ceph-mgr-modules-core_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-15T03:34:22.115 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking ceph-mgr-modules-core (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:34:22.273 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-15T03:34:22.292 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../030-python3-cherrypy3_8.9.1-6_all.deb ... 2024-09-15T03:34:22.331 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2024-09-15T03:34:22.478 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-15T03:34:22.495 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../029-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2024-09-15T03:34:22.536 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-15T03:34:22.734 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package python3-natsort. 2024-09-15T03:34:22.751 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../031-python3-natsort_7.0.1-1_all.deb ... 2024-09-15T03:34:22.784 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking python3-natsort (7.0.1-1) ... 2024-09-15T03:34:22.877 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-15T03:34:22.895 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../030-python3-cherrypy3_8.9.1-6_all.deb ... 2024-09-15T03:34:22.934 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2024-09-15T03:34:23.129 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package python3-jwt. 2024-09-15T03:34:23.151 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../032-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2024-09-15T03:34:23.187 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-15T03:34:23.251 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-natsort. 2024-09-15T03:34:23.271 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../031-python3-natsort_7.0.1-1_all.deb ... 2024-09-15T03:34:23.309 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-natsort (7.0.1-1) ... 2024-09-15T03:34:23.507 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package python3-logutils. 2024-09-15T03:34:23.526 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../033-python3-logutils_0.3.3-7_all.deb ... 2024-09-15T03:34:23.565 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking python3-logutils (0.3.3-7) ... 2024-09-15T03:34:23.621 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-jwt. 2024-09-15T03:34:23.637 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../032-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2024-09-15T03:34:23.679 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-15T03:34:23.893 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-15T03:34:23.912 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../034-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2024-09-15T03:34:23.951 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2024-09-15T03:34:23.974 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-logutils. 2024-09-15T03:34:23.989 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../033-python3-logutils_0.3.3-7_all.deb ... 2024-09-15T03:34:24.024 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-logutils (0.3.3-7) ... 2024-09-15T03:34:24.220 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package python3-mako. 2024-09-15T03:34:24.236 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../035-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2024-09-15T03:34:24.278 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-15T03:34:24.327 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-15T03:34:24.344 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../034-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2024-09-15T03:34:24.385 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2024-09-15T03:34:24.615 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-15T03:34:24.631 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../036-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-15T03:34:24.681 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-mako. 2024-09-15T03:34:24.690 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-15T03:34:24.700 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../035-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2024-09-15T03:34:24.740 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-15T03:34:24.993 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-15T03:34:25.008 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../037-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2024-09-15T03:34:25.051 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2024-09-15T03:34:25.051 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-15T03:34:25.070 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../036-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-15T03:34:25.109 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-15T03:34:25.362 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package python3-webob. 2024-09-15T03:34:25.378 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../038-python3-webob_1%3a1.8.5-2ubuntu0.1_all.deb ... 2024-09-15T03:34:25.412 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-15T03:34:25.421 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-15T03:34:25.429 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../037-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2024-09-15T03:34:25.470 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2024-09-15T03:34:25.771 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-webob. 2024-09-15T03:34:25.785 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package python3-soupsieve. 2024-09-15T03:34:25.790 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../038-python3-webob_1%3a1.8.5-2ubuntu0.1_all.deb ... 2024-09-15T03:34:25.803 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../039-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2024-09-15T03:34:25.832 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-15T03:34:25.844 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-15T03:34:26.147 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package python3-bs4. 2024-09-15T03:34:26.151 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-soupsieve. 2024-09-15T03:34:26.164 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../040-python3-bs4_4.8.2-1_all.deb ... 2024-09-15T03:34:26.167 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../039-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2024-09-15T03:34:26.201 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-15T03:34:26.205 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2024-09-15T03:34:26.471 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-bs4. 2024-09-15T03:34:26.487 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../040-python3-bs4_4.8.2-1_all.deb ... 2024-09-15T03:34:26.529 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2024-09-15T03:34:26.592 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package python3-waitress. 2024-09-15T03:34:26.610 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../041-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2024-09-15T03:34:26.653 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-15T03:34:26.924 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-waitress. 2024-09-15T03:34:26.941 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../041-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2024-09-15T03:34:26.970 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package python3-tempita. 2024-09-15T03:34:26.986 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-15T03:34:26.989 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../042-python3-tempita_0.5.2-6_all.deb ... 2024-09-15T03:34:27.028 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking python3-tempita (0.5.2-6) ... 2024-09-15T03:34:27.311 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-tempita. 2024-09-15T03:34:27.330 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../042-python3-tempita_0.5.2-6_all.deb ... 2024-09-15T03:34:27.364 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package python3-paste. 2024-09-15T03:34:27.369 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-tempita (0.5.2-6) ... 2024-09-15T03:34:27.382 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../043-python3-paste_3.4.0+dfsg1-1_all.deb ... 2024-09-15T03:34:27.422 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2024-09-15T03:34:27.629 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-paste. 2024-09-15T03:34:27.646 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../043-python3-paste_3.4.0+dfsg1-1_all.deb ... 2024-09-15T03:34:27.687 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2024-09-15T03:34:27.734 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-15T03:34:27.750 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../044-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2024-09-15T03:34:27.809 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2024-09-15T03:34:28.000 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-15T03:34:28.019 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../044-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2024-09-15T03:34:28.050 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2024-09-15T03:34:28.112 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-15T03:34:28.130 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../045-python3-pastedeploy_2.1.0-1_all.deb ... 2024-09-15T03:34:28.170 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2024-09-15T03:34:28.311 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-15T03:34:28.330 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../045-python3-pastedeploy_2.1.0-1_all.deb ... 2024-09-15T03:34:28.369 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2024-09-15T03:34:28.490 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package python3-webtest. 2024-09-15T03:34:28.507 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../046-python3-webtest_2.0.32-2_all.deb ... 2024-09-15T03:34:28.548 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking python3-webtest (2.0.32-2) ... 2024-09-15T03:34:28.655 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-webtest. 2024-09-15T03:34:28.675 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../046-python3-webtest_2.0.32-2_all.deb ... 2024-09-15T03:34:28.713 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-webtest (2.0.32-2) ... 2024-09-15T03:34:28.863 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package python3-pecan. 2024-09-15T03:34:28.879 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../047-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2024-09-15T03:34:28.921 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2024-09-15T03:34:29.050 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-pecan. 2024-09-15T03:34:29.066 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../047-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2024-09-15T03:34:29.108 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2024-09-15T03:34:29.251 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-15T03:34:29.270 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../048-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.2_all.deb ... 2024-09-15T03:34:29.301 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-15T03:34:29.455 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-15T03:34:29.474 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../048-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.2_all.deb ... 2024-09-15T03:34:29.513 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-15T03:34:29.652 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-15T03:34:29.668 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../049-libsqlite3-mod-ceph_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:34:29.712 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking libsqlite3-mod-ceph (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:34:29.901 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-15T03:34:29.917 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../049-libsqlite3-mod-ceph_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:34:29.959 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking libsqlite3-mod-ceph (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:34:30.072 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package ceph-mgr. 2024-09-15T03:34:30.090 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../050-ceph-mgr_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:34:30.121 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking ceph-mgr (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:34:30.312 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package ceph-mgr. 2024-09-15T03:34:30.328 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../050-ceph-mgr_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:34:30.370 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking ceph-mgr (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:34:30.633 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package ceph-mon. 2024-09-15T03:34:30.651 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../051-ceph-mon_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:34:30.691 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking ceph-mon (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:34:30.853 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package ceph-mon. 2024-09-15T03:34:30.871 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../051-ceph-mon_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:34:30.910 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking ceph-mon (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:34:31.554 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package ceph-osd. 2024-09-15T03:34:31.573 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../052-ceph-osd_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:34:31.612 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking ceph-osd (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:34:31.798 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package ceph-osd. 2024-09-15T03:34:31.816 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../052-ceph-osd_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:34:31.856 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking ceph-osd (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:34:33.831 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package ceph. 2024-09-15T03:34:33.847 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../053-ceph_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:34:33.922 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking ceph (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:34:34.134 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package ceph. 2024-09-15T03:34:34.152 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../053-ceph_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:34:34.192 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking ceph (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:34:34.250 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package ceph-fuse. 2024-09-15T03:34:34.267 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../054-ceph-fuse_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:34:34.309 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking ceph-fuse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:34:34.495 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package ceph-fuse. 2024-09-15T03:34:34.512 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../054-ceph-fuse_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:34:34.553 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking ceph-fuse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:34:34.770 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package ceph-mds. 2024-09-15T03:34:34.787 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../055-ceph-mds_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:34:34.828 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking ceph-mds (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:34:34.998 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package ceph-mds. 2024-09-15T03:34:35.016 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../055-ceph-mds_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:34:35.056 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking ceph-mds (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:34:35.356 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package cephadm. 2024-09-15T03:34:35.374 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../056-cephadm_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:34:35.406 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking cephadm (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:34:35.602 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package cephadm. 2024-09-15T03:34:35.620 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../056-cephadm_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:34:35.660 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking cephadm (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:34:35.710 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package python3-jinja2. 2024-09-15T03:34:35.727 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../057-python3-jinja2_2.10.1-2ubuntu0.3_all.deb ... 2024-09-15T03:34:35.769 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-15T03:34:36.004 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-jinja2. 2024-09-15T03:34:36.021 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../057-python3-jinja2_2.10.1-2ubuntu0.3_all.deb ... 2024-09-15T03:34:36.062 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-15T03:34:36.096 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-15T03:34:36.112 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../058-ceph-mgr-cephadm_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-15T03:34:36.154 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking ceph-mgr-cephadm (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:34:36.382 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-15T03:34:36.398 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../058-ceph-mgr-cephadm_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-15T03:34:36.441 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking ceph-mgr-cephadm (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:34:36.499 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-15T03:34:36.516 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../059-python3-repoze.lru_0.7-2_all.deb ... 2024-09-15T03:34:36.557 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-15T03:34:36.794 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-15T03:34:36.810 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../059-python3-repoze.lru_0.7-2_all.deb ... 2024-09-15T03:34:36.818 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package libjs-underscore. 2024-09-15T03:34:36.835 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../060-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2024-09-15T03:34:36.852 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-15T03:34:36.876 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-15T03:34:37.104 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package libjs-underscore. 2024-09-15T03:34:37.121 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../060-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2024-09-15T03:34:37.154 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package libjs-sphinxdoc. 2024-09-15T03:34:37.162 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-15T03:34:37.173 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../061-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2024-09-15T03:34:37.212 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-15T03:34:37.432 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package libjs-sphinxdoc. 2024-09-15T03:34:37.449 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../061-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2024-09-15T03:34:37.490 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-15T03:34:37.565 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package python3-routes. 2024-09-15T03:34:37.581 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../062-python3-routes_2.4.1-2_all.deb ... 2024-09-15T03:34:37.623 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking python3-routes (2.4.1-2) ... 2024-09-15T03:34:37.902 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-routes. 2024-09-15T03:34:37.918 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../062-python3-routes_2.4.1-2_all.deb ... 2024-09-15T03:34:37.959 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-routes (2.4.1-2) ... 2024-09-15T03:34:38.034 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-15T03:34:38.051 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../063-ceph-mgr-dashboard_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-15T03:34:38.092 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking ceph-mgr-dashboard (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:34:38.271 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-15T03:34:38.288 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../063-ceph-mgr-dashboard_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-15T03:34:38.329 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking ceph-mgr-dashboard (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:34:39.424 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package python3-decorator. 2024-09-15T03:34:39.442 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../064-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2024-09-15T03:34:39.482 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2024-09-15T03:34:39.635 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-decorator. 2024-09-15T03:34:39.652 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../064-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2024-09-15T03:34:39.726 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2024-09-15T03:34:39.793 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2024-09-15T03:34:39.809 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../065-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2024-09-15T03:34:39.851 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-15T03:34:40.029 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2024-09-15T03:34:40.045 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../065-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2024-09-15T03:34:40.087 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-15T03:34:40.179 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package python3-scipy. 2024-09-15T03:34:40.197 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../066-python3-scipy_1.3.3-3build1_amd64.deb ... 2024-09-15T03:34:40.237 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2024-09-15T03:34:40.407 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-scipy. 2024-09-15T03:34:40.424 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../066-python3-scipy_1.3.3-3build1_amd64.deb ... 2024-09-15T03:34:40.465 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2024-09-15T03:34:41.529 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package python3-sklearn-lib. 2024-09-15T03:34:41.549 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../067-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2024-09-15T03:34:41.587 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-15T03:34:41.793 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-sklearn-lib. 2024-09-15T03:34:41.809 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../067-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2024-09-15T03:34:41.851 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-15T03:34:42.132 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package python3-joblib. 2024-09-15T03:34:42.148 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../068-python3-joblib_0.14.0-3_all.deb ... 2024-09-15T03:34:42.190 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking python3-joblib (0.14.0-3) ... 2024-09-15T03:34:42.380 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-joblib. 2024-09-15T03:34:42.399 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../068-python3-joblib_0.14.0-3_all.deb ... 2024-09-15T03:34:42.438 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-joblib (0.14.0-3) ... 2024-09-15T03:34:42.560 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package python3-sklearn. 2024-09-15T03:34:42.576 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../069-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2024-09-15T03:34:42.618 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-15T03:34:42.825 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-sklearn. 2024-09-15T03:34:42.841 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../069-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2024-09-15T03:34:42.891 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-15T03:34:43.251 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-15T03:34:43.269 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../070-ceph-mgr-diskprediction-local_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-15T03:34:43.334 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking ceph-mgr-diskprediction-local (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:34:43.550 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-15T03:34:43.570 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../070-ceph-mgr-diskprediction-local_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-15T03:34:43.608 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking ceph-mgr-diskprediction-local (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:34:44.454 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package python3-cachetools. 2024-09-15T03:34:44.473 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../071-python3-cachetools_4.0.0-1_all.deb ... 2024-09-15T03:34:44.512 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2024-09-15T03:34:44.745 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-cachetools. 2024-09-15T03:34:44.764 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../071-python3-cachetools_4.0.0-1_all.deb ... 2024-09-15T03:34:44.802 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2024-09-15T03:34:44.822 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-15T03:34:44.842 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../072-python3-pyasn1_0.4.2-3build1_all.deb ... 2024-09-15T03:34:44.881 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2024-09-15T03:34:45.105 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-15T03:34:45.122 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../072-python3-pyasn1_0.4.2-3build1_all.deb ... 2024-09-15T03:34:45.163 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2024-09-15T03:34:45.217 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-15T03:34:45.235 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../073-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2024-09-15T03:34:45.275 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-15T03:34:45.475 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-15T03:34:45.495 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../073-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2024-09-15T03:34:45.542 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-15T03:34:45.612 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package python3-rsa. 2024-09-15T03:34:45.628 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../074-python3-rsa_4.0-3_all.deb ... 2024-09-15T03:34:45.670 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking python3-rsa (4.0-3) ... 2024-09-15T03:34:45.903 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-rsa. 2024-09-15T03:34:45.921 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../074-python3-rsa_4.0-3_all.deb ... 2024-09-15T03:34:45.932 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package python3-google-auth. 2024-09-15T03:34:45.948 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../075-python3-google-auth_1.5.1-2_all.deb ... 2024-09-15T03:34:45.961 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-rsa (4.0-3) ... 2024-09-15T03:34:45.990 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2024-09-15T03:34:46.264 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-google-auth. 2024-09-15T03:34:46.283 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../075-python3-google-auth_1.5.1-2_all.deb ... 2024-09-15T03:34:46.301 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package python3-blinker. 2024-09-15T03:34:46.321 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../076-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2024-09-15T03:34:46.322 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2024-09-15T03:34:46.359 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-15T03:34:46.645 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package python3-oauthlib. 2024-09-15T03:34:46.662 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../077-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2024-09-15T03:34:46.668 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-blinker. 2024-09-15T03:34:46.687 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../076-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2024-09-15T03:34:46.703 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-15T03:34:46.726 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-15T03:34:47.029 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-oauthlib. 2024-09-15T03:34:47.041 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-15T03:34:47.047 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../077-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2024-09-15T03:34:47.061 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../078-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2024-09-15T03:34:47.087 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-15T03:34:47.099 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-15T03:34:47.399 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-15T03:34:47.418 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package python3-websocket. 2024-09-15T03:34:47.419 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../078-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2024-09-15T03:34:47.435 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../079-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2024-09-15T03:34:47.490 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-15T03:34:47.535 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2024-09-15T03:34:47.828 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-websocket. 2024-09-15T03:34:47.847 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-15T03:34:47.847 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../079-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2024-09-15T03:34:47.864 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../080-python3-kubernetes_7.0.0~a1-2_all.deb ... 2024-09-15T03:34:47.886 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2024-09-15T03:34:47.904 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2024-09-15T03:34:48.514 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-15T03:34:48.535 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../080-python3-kubernetes_7.0.0~a1-2_all.deb ... 2024-09-15T03:34:48.572 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2024-09-15T03:34:48.965 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-15T03:34:48.985 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../081-ceph-mgr-k8sevents_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-15T03:34:49.023 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking ceph-mgr-k8sevents (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:34:49.231 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-15T03:34:49.251 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../081-ceph-mgr-k8sevents_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-15T03:34:49.289 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking ceph-mgr-k8sevents (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:34:49.318 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-15T03:34:49.338 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../082-libonig5_6.9.4-1_amd64.deb ... 2024-09-15T03:34:49.376 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2024-09-15T03:34:49.634 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-15T03:34:49.652 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../082-libonig5_6.9.4-1_amd64.deb ... 2024-09-15T03:34:49.701 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2024-09-15T03:34:49.762 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-15T03:34:49.779 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../083-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-15T03:34:49.820 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-15T03:34:50.062 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-15T03:34:50.078 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../083-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-15T03:34:50.098 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package jq. 2024-09-15T03:34:50.116 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../084-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-15T03:34:50.120 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-15T03:34:50.182 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2024-09-15T03:34:50.385 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package jq. 2024-09-15T03:34:50.403 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../084-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-15T03:34:50.418 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package socat. 2024-09-15T03:34:50.435 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../085-socat_1.7.3.3-2_amd64.deb ... 2024-09-15T03:34:50.443 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2024-09-15T03:34:50.467 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking socat (1.7.3.3-2) ... 2024-09-15T03:34:50.721 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package socat. 2024-09-15T03:34:50.739 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../085-socat_1.7.3.3-2_amd64.deb ... 2024-09-15T03:34:50.766 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package xmlstarlet. 2024-09-15T03:34:50.779 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking socat (1.7.3.3-2) ... 2024-09-15T03:34:50.785 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../086-xmlstarlet_1.6.1-2build1_amd64.deb ... 2024-09-15T03:34:50.841 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2024-09-15T03:34:51.083 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package xmlstarlet. 2024-09-15T03:34:51.100 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../086-xmlstarlet_1.6.1-2build1_amd64.deb ... 2024-09-15T03:34:51.141 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2024-09-15T03:34:51.196 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package ceph-test. 2024-09-15T03:34:51.216 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../087-ceph-test_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:34:51.254 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking ceph-test (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:34:51.488 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package ceph-test. 2024-09-15T03:34:51.505 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../087-ceph-test_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:34:51.546 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking ceph-test (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:34:55.890 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package ceph-volume. 2024-09-15T03:34:55.910 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../088-ceph-volume_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-15T03:34:55.942 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking ceph-volume (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:34:56.235 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package ceph-volume. 2024-09-15T03:34:56.238 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package fonts-mathjax. 2024-09-15T03:34:56.251 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../088-ceph-volume_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-15T03:34:56.255 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../089-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-15T03:34:56.293 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking ceph-volume (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:34:56.313 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-15T03:34:56.646 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package fonts-mathjax. 2024-09-15T03:34:56.667 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../089-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-15T03:34:56.703 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-15T03:34:56.838 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-15T03:34:56.851 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../090-libcephfs-dev_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:34:56.892 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking libcephfs-dev (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:34:57.232 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-15T03:34:57.248 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../090-libcephfs-dev_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:34:57.291 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking libcephfs-dev (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:34:57.312 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package nvme-cli. 2024-09-15T03:34:57.329 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../091-nvme-cli_1.9-1ubuntu0.2_amd64.deb ... 2024-09-15T03:34:57.388 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking nvme-cli (1.9-1ubuntu0.2) ... 2024-09-15T03:34:57.704 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package nvme-cli. 2024-09-15T03:34:57.721 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../091-nvme-cli_1.9-1ubuntu0.2_amd64.deb ... 2024-09-15T03:34:57.733 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package libjs-mathjax. 2024-09-15T03:34:57.753 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../092-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-15T03:34:57.780 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking nvme-cli (1.9-1ubuntu0.2) ... 2024-09-15T03:34:57.791 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-15T03:34:58.102 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package libjs-mathjax. 2024-09-15T03:34:58.118 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../092-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-15T03:34:58.160 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-15T03:34:59.466 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package python-natsort-doc. 2024-09-15T03:34:59.485 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../093-python-natsort-doc_7.0.1-1_all.deb ... 2024-09-15T03:34:59.524 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2024-09-15T03:34:59.760 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python-natsort-doc. 2024-09-15T03:34:59.781 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../093-python-natsort-doc_7.0.1-1_all.deb ... 2024-09-15T03:34:59.819 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2024-09-15T03:34:59.885 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package python3-atomicwrites. 2024-09-15T03:34:59.903 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../094-python3-atomicwrites_1.1.5-2build1_all.deb ... 2024-09-15T03:34:59.943 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2024-09-15T03:35:00.155 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-atomicwrites. 2024-09-15T03:35:00.171 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../094-python3-atomicwrites_1.1.5-2build1_all.deb ... 2024-09-15T03:35:00.212 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2024-09-15T03:35:00.221 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package python3-attr. 2024-09-15T03:35:00.242 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../095-python3-attr_19.3.0-2_all.deb ... 2024-09-15T03:35:00.270 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking python3-attr (19.3.0-2) ... 2024-09-15T03:35:00.541 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-attr. 2024-09-15T03:35:00.560 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../095-python3-attr_19.3.0-2_all.deb ... 2024-09-15T03:35:00.590 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-attr (19.3.0-2) ... 2024-09-15T03:35:00.598 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package python3-webencodings. 2024-09-15T03:35:00.616 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../096-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2024-09-15T03:35:00.656 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-15T03:35:00.910 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-webencodings. 2024-09-15T03:35:00.928 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../096-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2024-09-15T03:35:00.959 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package python3-html5lib. 2024-09-15T03:35:00.967 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-15T03:35:00.978 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../097-python3-html5lib_1.0.1-2_all.deb ... 2024-09-15T03:35:01.025 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2024-09-15T03:35:01.262 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-html5lib. 2024-09-15T03:35:01.281 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../097-python3-html5lib_1.0.1-2_all.deb ... 2024-09-15T03:35:01.320 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2024-09-15T03:35:01.371 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package python3-lxml:amd64. 2024-09-15T03:35:01.388 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../098-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2024-09-15T03:35:01.428 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-15T03:35:01.682 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-lxml:amd64. 2024-09-15T03:35:01.699 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../098-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2024-09-15T03:35:01.740 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-15T03:35:01.924 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package python3-pastescript. 2024-09-15T03:35:01.943 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../099-python3-pastescript_2.0.2-4_all.deb ... 2024-09-15T03:35:01.982 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-15T03:35:02.202 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-pastescript. 2024-09-15T03:35:02.218 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../099-python3-pastescript_2.0.2-4_all.deb ... 2024-09-15T03:35:02.259 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-15T03:35:02.285 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package python3-pluggy. 2024-09-15T03:35:02.305 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../100-python3-pluggy_0.13.0-2_all.deb ... 2024-09-15T03:35:02.368 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2024-09-15T03:35:02.596 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-pluggy. 2024-09-15T03:35:02.616 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../100-python3-pluggy_0.13.0-2_all.deb ... 2024-09-15T03:35:02.654 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2024-09-15T03:35:02.688 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package python3-psutil. 2024-09-15T03:35:02.707 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../101-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2024-09-15T03:35:02.746 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2024-09-15T03:35:02.966 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-psutil. 2024-09-15T03:35:02.982 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../101-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2024-09-15T03:35:03.024 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2024-09-15T03:35:03.090 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package python3-py. 2024-09-15T03:35:03.110 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../102-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2024-09-15T03:35:03.148 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2024-09-15T03:35:03.343 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-py. 2024-09-15T03:35:03.364 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../102-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2024-09-15T03:35:03.401 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2024-09-15T03:35:03.476 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-15T03:35:03.496 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../103-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2024-09-15T03:35:03.534 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-15T03:35:03.754 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-15T03:35:03.771 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../103-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2024-09-15T03:35:03.812 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-15T03:35:03.871 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-15T03:35:03.887 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../104-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2024-09-15T03:35:03.920 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-15T03:35:04.115 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-15T03:35:04.136 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../104-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2024-09-15T03:35:04.173 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-15T03:35:04.215 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package python3-pytest. 2024-09-15T03:35:04.232 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../105-python3-pytest_4.6.9-1_all.deb ... 2024-09-15T03:35:04.273 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking python3-pytest (4.6.9-1) ... 2024-09-15T03:35:04.501 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-pytest. 2024-09-15T03:35:04.520 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../105-python3-pytest_4.6.9-1_all.deb ... 2024-09-15T03:35:04.559 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-pytest (4.6.9-1) ... 2024-09-15T03:35:04.648 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package python3-simplejson. 2024-09-15T03:35:04.668 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../106-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2024-09-15T03:35:04.697 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-15T03:35:04.904 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-simplejson. 2024-09-15T03:35:04.922 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../106-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2024-09-15T03:35:04.950 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-15T03:35:04.962 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-15T03:35:04.967 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../107-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2024-09-15T03:35:05.008 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-15T03:35:05.240 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-15T03:35:05.257 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../107-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2024-09-15T03:35:05.298 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-15T03:35:05.754 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package radosgw. 2024-09-15T03:35:05.774 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../108-radosgw_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:35:05.812 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking radosgw (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:35:06.052 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package radosgw. 2024-09-15T03:35:06.069 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../108-radosgw_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:35:06.110 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking radosgw (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:35:07.582 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package rbd-fuse. 2024-09-15T03:35:07.599 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../109-rbd-fuse_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:35:07.640 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking rbd-fuse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:35:07.888 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package rbd-fuse. 2024-09-15T03:35:07.908 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../109-rbd-fuse_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:35:07.946 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking rbd-fuse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:35:07.970 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up python3-attr (19.3.0-2) ... 2024-09-15T03:35:08.240 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-attr (19.3.0-2) ... 2024-09-15T03:35:08.281 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2024-09-15T03:35:08.407 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-15T03:35:08.529 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2024-09-15T03:35:08.566 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up nvme-cli (1.9-1ubuntu0.2) ... 2024-09-15T03:35:08.655 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-15T03:35:08.759 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up cephadm (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:35:08.810 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up nvme-cli (1.9-1ubuntu0.2) ... 2024-09-15T03:35:09.007 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up cephadm (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:35:09.362 INFO:teuthology.orchestra.run.smithi177.stdout:Adding system user cephadm....done 2024-09-15T03:35:09.523 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-15T03:35:09.610 INFO:teuthology.orchestra.run.smithi110.stdout:Adding system user cephadm....done 2024-09-15T03:35:09.642 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-15T03:35:09.772 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-15T03:35:09.906 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-15T03:35:09.969 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-15T03:35:10.242 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-15T03:35:10.270 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-15T03:35:10.395 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up liboath0 (2.6.1-1.3) ... 2024-09-15T03:35:10.528 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-15T03:35:10.555 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2024-09-15T03:35:10.653 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up liboath0 (2.6.1-1.3) ... 2024-09-15T03:35:10.780 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2024-09-15T03:35:10.882 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up python3-joblib (0.14.0-3) ... 2024-09-15T03:35:11.098 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-joblib (0.14.0-3) ... 2024-09-15T03:35:11.284 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-15T03:35:11.500 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-15T03:35:11.551 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up python3-cachetools (4.0.0-1) ... 2024-09-15T03:35:11.768 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-cachetools (4.0.0-1) ... 2024-09-15T03:35:11.853 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-15T03:35:12.060 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-15T03:35:12.145 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-15T03:35:12.336 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-15T03:35:12.478 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up python3-ceph-argparse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:35:12.648 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-ceph-argparse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:35:12.789 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-15T03:35:12.915 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2024-09-15T03:35:12.940 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-15T03:35:13.041 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2024-09-15T03:35:13.216 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-15T03:35:13.301 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-15T03:35:13.484 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-15T03:35:13.568 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-15T03:35:13.585 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2024-09-15T03:35:13.694 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2024-09-15T03:35:13.969 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up python3-natsort (7.0.1-1) ... 2024-09-15T03:35:14.070 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-natsort (7.0.1-1) ... 2024-09-15T03:35:14.273 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2024-09-15T03:35:14.355 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2024-09-15T03:35:14.563 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-15T03:35:14.664 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-15T03:35:14.915 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2024-09-15T03:35:15.007 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2024-09-15T03:35:15.207 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-15T03:35:15.267 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-15T03:35:15.558 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up python3-tempita (0.5.2-6) ... 2024-09-15T03:35:15.576 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-tempita (0.5.2-6) ... 2024-09-15T03:35:15.860 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up python3-packaging (20.3-1) ... 2024-09-15T03:35:15.869 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-packaging (20.3-1) ... 2024-09-15T03:35:16.136 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2024-09-15T03:35:16.153 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2024-09-15T03:35:16.262 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-15T03:35:16.279 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-15T03:35:16.396 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-15T03:35:16.423 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-15T03:35:16.689 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2024-09-15T03:35:16.732 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2024-09-15T03:35:17.090 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-15T03:35:17.117 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-15T03:35:17.451 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-15T03:35:17.483 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-15T03:35:17.820 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up socat (1.7.3.3-2) ... 2024-09-15T03:35:17.827 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up socat (1.7.3.3-2) ... 2024-09-15T03:35:17.946 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-ceph-common (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:35:17.995 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up python3-ceph-common (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:35:18.264 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-15T03:35:18.304 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-15T03:35:18.431 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-html5lib (1.0.1-2) ... 2024-09-15T03:35:18.439 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up python3-html5lib (1.0.1-2) ... 2024-09-15T03:35:18.766 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-15T03:35:18.807 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-15T03:35:18.868 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2024-09-15T03:35:18.933 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2024-09-15T03:35:18.976 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-pluggy (0.13.0-2) ... 2024-09-15T03:35:19.042 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up python3-pluggy (0.13.0-2) ... 2024-09-15T03:35:19.653 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-15T03:35:19.693 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-15T03:35:19.979 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-15T03:35:20.003 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-15T03:35:20.105 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-15T03:35:20.129 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-15T03:35:20.231 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2024-09-15T03:35:20.263 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2024-09-15T03:35:20.566 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2024-09-15T03:35:20.615 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2024-09-15T03:35:20.867 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-logutils (0.3.3-7) ... 2024-09-15T03:35:20.891 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up python3-logutils (0.3.3-7) ... 2024-09-15T03:35:21.126 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-15T03:35:21.183 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-15T03:35:21.237 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-15T03:35:21.334 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-15T03:35:21.512 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-prettytable (0.7.2-5) ... 2024-09-15T03:35:21.619 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up python3-prettytable (0.7.2-5) ... 2024-09-15T03:35:21.821 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-15T03:35:21.945 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-15T03:35:22.131 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2024-09-15T03:35:22.246 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2024-09-15T03:35:22.515 INFO:teuthology.orchestra.run.smithi110.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2024-09-15T03:35:22.606 INFO:teuthology.orchestra.run.smithi177.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2024-09-15T03:35:22.758 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2024-09-15T03:35:22.859 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-15T03:35:22.873 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2024-09-15T03:35:22.991 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-15T03:35:23.152 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2024-09-15T03:35:23.309 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2024-09-15T03:35:23.495 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-15T03:35:23.655 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-15T03:35:23.668 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-15T03:35:23.780 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-15T03:35:23.794 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-15T03:35:23.945 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-15T03:35:24.124 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-15T03:35:24.280 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-15T03:35:24.408 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up librados2 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:35:24.517 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2024-09-15T03:35:24.564 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up librados2 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:35:24.715 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2024-09-15T03:35:25.973 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up libsqlite3-mod-ceph (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:35:26.099 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-15T03:35:26.170 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up libsqlite3-mod-ceph (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:35:26.225 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-pytest (4.6.9-1) ... 2024-09-15T03:35:26.305 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-15T03:35:26.430 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up python3-pytest (4.6.9-1) ... 2024-09-15T03:35:26.593 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2024-09-15T03:35:26.782 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2024-09-15T03:35:26.861 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2024-09-15T03:35:27.050 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2024-09-15T03:35:27.162 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-15T03:35:27.288 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-bs4 (4.8.2-1) ... 2024-09-15T03:35:27.342 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-15T03:35:27.468 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up python3-bs4 (4.8.2-1) ... 2024-09-15T03:35:27.648 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up libcephfs2 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:35:27.774 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-15T03:35:27.803 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up libcephfs2 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:35:27.929 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-15T03:35:28.092 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up libradosstriper1 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:35:28.195 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-15T03:35:28.264 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up libradosstriper1 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:35:28.390 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-15T03:35:28.502 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-15T03:35:28.708 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-15T03:35:29.566 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up librbd1 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:35:29.684 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-15T03:35:29.738 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up librbd1 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:35:29.785 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2024-09-15T03:35:29.864 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-15T03:35:29.886 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2024-09-15T03:35:30.003 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-rsa (4.0-3) ... 2024-09-15T03:35:30.006 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2024-09-15T03:35:30.132 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2024-09-15T03:35:30.258 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up python3-rsa (4.0-3) ... 2024-09-15T03:35:30.296 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-15T03:35:30.422 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-webtest (2.0.32-2) ... 2024-09-15T03:35:30.567 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-15T03:35:30.718 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up python3-webtest (2.0.32-2) ... 2024-09-15T03:35:30.749 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up ceph-mgr-modules-core (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:35:30.874 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up ceph-fuse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:35:31.019 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up ceph-mgr-modules-core (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:35:31.098 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-15T03:35:31.098 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-15T03:35:31.154 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up ceph-fuse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:35:31.378 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-15T03:35:31.378 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-15T03:35:31.567 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up libcephfs-dev (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:35:31.778 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-15T03:35:31.838 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up libcephfs-dev (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:35:32.063 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-15T03:35:32.074 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-15T03:35:32.367 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2024-09-15T03:35:32.376 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-15T03:35:32.711 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2024-09-15T03:35:32.728 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-rados (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:35:32.854 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up librgw2 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:35:32.980 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-rbd (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:35:33.065 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up python3-rados (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:35:33.109 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up rbd-fuse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:35:33.173 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up librgw2 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:35:33.232 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-routes (2.4.1-2) ... 2024-09-15T03:35:33.299 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up python3-rbd (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:35:33.458 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up rbd-fuse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:35:33.559 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-google-auth (1.5.1-2) ... 2024-09-15T03:35:33.584 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up python3-routes (2.4.1-2) ... 2024-09-15T03:35:33.852 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up python3-google-auth (1.5.1-2) ... 2024-09-15T03:35:33.868 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-rgw (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:35:33.994 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-cephfs (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:35:34.120 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2024-09-15T03:35:34.128 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up python3-rgw (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:35:34.262 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up python3-cephfs (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:35:34.389 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2024-09-15T03:35:35.024 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up ceph-common (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:35:35.275 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up ceph-common (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:35:35.601 INFO:teuthology.orchestra.run.smithi110.stdout:Adding group ceph....done 2024-09-15T03:35:35.853 INFO:teuthology.orchestra.run.smithi177.stdout:Adding group ceph....done 2024-09-15T03:35:35.978 INFO:teuthology.orchestra.run.smithi110.stdout:Adding system user ceph....done 2024-09-15T03:35:36.154 INFO:teuthology.orchestra.run.smithi110.stdout:Setting system user ceph properties....done 2024-09-15T03:35:36.179 INFO:teuthology.orchestra.run.smithi110.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-15T03:35:36.237 INFO:teuthology.orchestra.run.smithi177.stdout:Adding system user ceph....done 2024-09-15T03:35:36.327 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-15T03:35:36.397 INFO:teuthology.orchestra.run.smithi177.stdout:Setting system user ceph properties....done 2024-09-15T03:35:36.423 INFO:teuthology.orchestra.run.smithi177.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-15T03:35:36.571 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-15T03:35:36.646 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-15T03:35:36.880 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-15T03:35:37.101 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up ceph-test (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:35:37.226 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up radosgw (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:35:37.335 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up ceph-test (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:35:37.494 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up radosgw (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:35:37.502 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-15T03:35:37.502 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-15T03:35:37.769 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-15T03:35:37.769 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-15T03:35:38.170 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up ceph-base (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:35:38.404 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up ceph-base (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:35:38.535 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-15T03:35:38.781 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-15T03:35:38.989 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up ceph-mds (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:35:39.221 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-15T03:35:39.222 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-15T03:35:39.247 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up ceph-mds (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:35:39.464 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-15T03:35:39.464 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-15T03:35:39.657 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up ceph-mgr (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:35:39.883 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-15T03:35:39.883 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-15T03:35:39.924 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up ceph-mgr (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:35:40.152 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-15T03:35:40.152 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-15T03:35:40.325 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up ceph-osd (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:35:40.609 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up ceph-osd (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:35:40.838 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-15T03:35:40.838 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-15T03:35:41.153 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-15T03:35:41.153 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-15T03:35:41.301 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up ceph-mgr-k8sevents (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:35:41.447 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up ceph-mgr-diskprediction-local (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:35:41.595 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up ceph-mon (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:35:41.619 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up ceph-mgr-k8sevents (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:35:41.753 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up ceph-mgr-diskprediction-local (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:35:41.822 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-15T03:35:41.822 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-15T03:35:41.896 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up ceph-mon (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:35:42.125 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-15T03:35:42.125 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-15T03:35:42.272 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up ceph-mgr-cephadm (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:35:42.414 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up ceph (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:35:42.557 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up ceph-mgr-dashboard (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:35:42.572 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up ceph-mgr-cephadm (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:35:42.698 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up ceph (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:35:42.699 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up ceph-volume (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:35:42.827 INFO:teuthology.orchestra.run.smithi110.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-09-15T03:35:42.841 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up ceph-mgr-dashboard (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:35:42.961 INFO:teuthology.orchestra.run.smithi110.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-09-15T03:35:42.983 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up ceph-volume (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:35:43.110 INFO:teuthology.orchestra.run.smithi177.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-09-15T03:35:43.246 INFO:teuthology.orchestra.run.smithi110.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-09-15T03:35:43.261 INFO:teuthology.orchestra.run.smithi177.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-09-15T03:35:43.538 INFO:teuthology.orchestra.run.smithi177.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-09-15T03:35:46.041 INFO:teuthology.orchestra.run.smithi110.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2024-09-15T03:35:46.441 INFO:teuthology.orchestra.run.smithi177.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2024-09-15T03:35:46.605 INFO:teuthology.orchestra.run.smithi110.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-15T03:35:46.608 DEBUG:teuthology.orchestra.run.smithi110:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install sqlite3 2024-09-15T03:35:46.682 INFO:teuthology.orchestra.run.smithi110.stdout:Reading package lists... 2024-09-15T03:35:46.901 INFO:teuthology.orchestra.run.smithi110.stdout:Building dependency tree... 2024-09-15T03:35:46.901 INFO:teuthology.orchestra.run.smithi110.stdout:Reading state information... 2024-09-15T03:35:47.056 INFO:teuthology.orchestra.run.smithi177.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-15T03:35:47.059 DEBUG:teuthology.orchestra.run.smithi177:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install sqlite3 2024-09-15T03:35:47.110 INFO:teuthology.orchestra.run.smithi110.stdout:The following packages were automatically installed and are no longer required: 2024-09-15T03:35:47.111 INFO:teuthology.orchestra.run.smithi110.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-09-15T03:35:47.111 INFO:teuthology.orchestra.run.smithi110.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-15T03:35:47.112 INFO:teuthology.orchestra.run.smithi110.stdout:The following additional packages will be installed: 2024-09-15T03:35:47.113 INFO:teuthology.orchestra.run.smithi110.stdout: libsqlite3-0 2024-09-15T03:35:47.114 INFO:teuthology.orchestra.run.smithi110.stdout:Suggested packages: 2024-09-15T03:35:47.114 INFO:teuthology.orchestra.run.smithi110.stdout: sqlite3-doc 2024-09-15T03:35:47.136 INFO:teuthology.orchestra.run.smithi177.stdout:Reading package lists... 2024-09-15T03:35:47.150 INFO:teuthology.orchestra.run.smithi110.stdout:The following NEW packages will be installed: 2024-09-15T03:35:47.150 INFO:teuthology.orchestra.run.smithi110.stdout: sqlite3 2024-09-15T03:35:47.151 INFO:teuthology.orchestra.run.smithi110.stdout:The following packages will be upgraded: 2024-09-15T03:35:47.152 INFO:teuthology.orchestra.run.smithi110.stdout: libsqlite3-0 2024-09-15T03:35:47.205 INFO:teuthology.orchestra.run.smithi110.stdout:1 upgraded, 1 newly installed, 0 to remove and 294 not upgraded. 2024-09-15T03:35:47.205 INFO:teuthology.orchestra.run.smithi110.stdout:Need to get 1,409 kB of archives. 2024-09-15T03:35:47.205 INFO:teuthology.orchestra.run.smithi110.stdout:After this operation, 2,803 kB of additional disk space will be used. 2024-09-15T03:35:47.205 INFO:teuthology.orchestra.run.smithi110.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libsqlite3-0 amd64 3.31.1-4ubuntu0.6 [549 kB] 2024-09-15T03:35:47.321 INFO:teuthology.orchestra.run.smithi110.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 sqlite3 amd64 3.31.1-4ubuntu0.6 [860 kB] 2024-09-15T03:35:47.356 INFO:teuthology.orchestra.run.smithi177.stdout:Building dependency tree... 2024-09-15T03:35:47.356 INFO:teuthology.orchestra.run.smithi177.stdout:Reading state information... 2024-09-15T03:35:47.565 INFO:teuthology.orchestra.run.smithi177.stdout:The following packages were automatically installed and are no longer required: 2024-09-15T03:35:47.566 INFO:teuthology.orchestra.run.smithi177.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-09-15T03:35:47.566 INFO:teuthology.orchestra.run.smithi177.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-15T03:35:47.567 INFO:teuthology.orchestra.run.smithi177.stdout:The following additional packages will be installed: 2024-09-15T03:35:47.567 INFO:teuthology.orchestra.run.smithi177.stdout: libsqlite3-0 2024-09-15T03:35:47.569 INFO:teuthology.orchestra.run.smithi177.stdout:Suggested packages: 2024-09-15T03:35:47.569 INFO:teuthology.orchestra.run.smithi177.stdout: sqlite3-doc 2024-09-15T03:35:47.604 INFO:teuthology.orchestra.run.smithi177.stdout:The following NEW packages will be installed: 2024-09-15T03:35:47.604 INFO:teuthology.orchestra.run.smithi177.stdout: sqlite3 2024-09-15T03:35:47.605 INFO:teuthology.orchestra.run.smithi177.stdout:The following packages will be upgraded: 2024-09-15T03:35:47.606 INFO:teuthology.orchestra.run.smithi177.stdout: libsqlite3-0 2024-09-15T03:35:47.723 INFO:teuthology.orchestra.run.smithi110.stdout:Fetched 1,409 kB in 0s (7,613 kB/s) 2024-09-15T03:35:47.789 INFO:teuthology.orchestra.run.smithi177.stdout:1 upgraded, 1 newly installed, 0 to remove and 294 not upgraded. 2024-09-15T03:35:47.789 INFO:teuthology.orchestra.run.smithi177.stdout:Need to get 1,409 kB of archives. 2024-09-15T03:35:47.789 INFO:teuthology.orchestra.run.smithi177.stdout:After this operation, 2,803 kB of additional disk space will be used. 2024-09-15T03:35:47.789 INFO:teuthology.orchestra.run.smithi177.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libsqlite3-0 amd64 3.31.1-4ubuntu0.6 [549 kB] 2024-09-15T03:35:47.952 INFO:teuthology.orchestra.run.smithi110.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152912 files and directories currently installed.) 2024-09-15T03:35:47.957 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../libsqlite3-0_3.31.1-4ubuntu0.6_amd64.deb ... 2024-09-15T03:35:48.137 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking libsqlite3-0:amd64 (3.31.1-4ubuntu0.6) over (3.31.1-4ubuntu0.3) ... 2024-09-15T03:35:48.336 INFO:teuthology.orchestra.run.smithi177.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 sqlite3 amd64 3.31.1-4ubuntu0.6 [860 kB] 2024-09-15T03:35:48.525 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package sqlite3. 2024-09-15T03:35:48.544 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../sqlite3_3.31.1-4ubuntu0.6_amd64.deb ... 2024-09-15T03:35:48.583 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking sqlite3 (3.31.1-4ubuntu0.6) ... 2024-09-15T03:35:48.827 INFO:teuthology.orchestra.run.smithi177.stdout:Fetched 1,409 kB in 1s (1,687 kB/s) 2024-09-15T03:35:48.993 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up libsqlite3-0:amd64 (3.31.1-4ubuntu0.6) ... 2024-09-15T03:35:49.055 INFO:teuthology.orchestra.run.smithi177.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152912 files and directories currently installed.) 2024-09-15T03:35:49.059 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../libsqlite3-0_3.31.1-4ubuntu0.6_amd64.deb ... 2024-09-15T03:35:49.113 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up sqlite3 (3.31.1-4ubuntu0.6) ... 2024-09-15T03:35:49.237 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking libsqlite3-0:amd64 (3.31.1-4ubuntu0.6) over (3.31.1-4ubuntu0.3) ... 2024-09-15T03:35:49.239 INFO:teuthology.orchestra.run.smithi110.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-09-15T03:35:49.574 INFO:teuthology.orchestra.run.smithi177.stdout:Selecting previously unselected package sqlite3. 2024-09-15T03:35:49.591 INFO:teuthology.orchestra.run.smithi177.stdout:Preparing to unpack .../sqlite3_3.31.1-4ubuntu0.6_amd64.deb ... 2024-09-15T03:35:49.623 INFO:teuthology.orchestra.run.smithi177.stdout:Unpacking sqlite3 (3.31.1-4ubuntu0.6) ... 2024-09-15T03:35:49.717 INFO:teuthology.orchestra.run.smithi110.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-09-15T03:35:50.023 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up libsqlite3-0:amd64 (3.31.1-4ubuntu0.6) ... 2024-09-15T03:35:50.175 INFO:teuthology.orchestra.run.smithi110.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-15T03:35:50.179 DEBUG:teuthology.parallel:result is None 2024-09-15T03:35:50.504 INFO:teuthology.orchestra.run.smithi177.stdout:Setting up sqlite3 (3.31.1-4ubuntu0.6) ... 2024-09-15T03:35:51.062 INFO:teuthology.orchestra.run.smithi177.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-09-15T03:35:51.576 INFO:teuthology.orchestra.run.smithi177.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-09-15T03:35:52.022 INFO:teuthology.orchestra.run.smithi177.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-15T03:35:52.026 DEBUG:teuthology.parallel:result is None 2024-09-15T03:35:52.026 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=d75f23820806da622f561ee5ee7551015285993f 2024-09-15T03:35:52.181 DEBUG:teuthology.orchestra.run.smithi110:> dpkg-query -W -f '${Version}' ceph 2024-09-15T03:35:52.206 INFO:teuthology.orchestra.run.smithi110.stdout:17.2.7-1643-gd75f2382-1focal 2024-09-15T03:35:52.206 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1643-gd75f2382-1focal 2024-09-15T03:35:52.206 INFO:teuthology.task.install:The correct ceph version 17.2.7-1643-gd75f2382-1focal is installed. 2024-09-15T03:35:52.208 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=d75f23820806da622f561ee5ee7551015285993f 2024-09-15T03:35:52.358 DEBUG:teuthology.orchestra.run.smithi177:> dpkg-query -W -f '${Version}' ceph 2024-09-15T03:35:52.381 INFO:teuthology.orchestra.run.smithi177.stdout:17.2.7-1643-gd75f2382-1focal 2024-09-15T03:35:52.382 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1643-gd75f2382-1focal 2024-09-15T03:35:52.382 INFO:teuthology.task.install:The correct ceph version 17.2.7-1643-gd75f2382-1focal is installed. 2024-09-15T03:35:52.386 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-15T03:35:52.386 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-15T03:35:52.386 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-15T03:35:52.403 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-15T03:35:52.403 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-15T03:35:52.438 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-15T03:35:52.438 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-15T03:35:52.439 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/usr/bin/daemon-helper 2024-09-15T03:35:52.460 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-15T03:35:52.474 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-15T03:35:52.474 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/usr/bin/daemon-helper 2024-09-15T03:35:52.489 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-15T03:35:52.498 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-15T03:35:52.498 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-15T03:35:52.498 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-15T03:35:52.513 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-15T03:35:52.572 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-15T03:35:52.572 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-15T03:35:52.587 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-15T03:35:52.645 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-15T03:35:52.645 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-15T03:35:52.645 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/usr/bin/stdin-killer 2024-09-15T03:35:52.661 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-15T03:35:52.670 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-15T03:35:52.670 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/usr/bin/stdin-killer 2024-09-15T03:35:52.686 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-15T03:35:52.697 INFO:teuthology.run_tasks:Running task ceph... 2024-09-15T03:35:52.770 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-15T03:35:52.772 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod 777 /var/log/ceph 2024-09-15T03:35:52.774 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 777 /var/log/ceph 2024-09-15T03:35:52.790 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-15T03:35:52.791 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-15T03:35:52.793 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-15T03:35:52.806 INFO:tasks.ceph:Creating extra log directories... 2024-09-15T03:35:52.806 DEBUG:teuthology.orchestra.run.smithi110:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-15T03:35:52.847 DEBUG:teuthology.orchestra.run.smithi177:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-15T03:35:52.876 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-15T03:35:52.876 INFO:tasks.ceph:config {'conf': {'client': {'debug cephsqlite': 20, 'debug client': 20, 'debug ms': 1}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject socket failures': 5000, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'osd': {'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-15T03:35:52.876 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905284', 'branch': 'wip-bharath9-testing-2024-09-13-1444-quincy', 'description': 'rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} tasks/libcephsqlite}', 'email': 'skanta@redhat.com', 'first_in_suite': False, 'job_id': '7905284', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'volumes': {'count': 4, 'size': 10}}], 'os_type': 'ubuntu', 'os_version': '20.04', 'overrides': {'admin_socket': {'branch': 'wip-bharath9-testing-2024-09-13-1444-quincy'}, 'ceph': {'conf': {'client': {'debug cephsqlite': 20, 'debug client': 20, 'debug ms': 1}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject socket failures': 5000, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'osd': {'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled'], 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-bharath9-testing-2024-09-13-1444-quincy', 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}}, 'owner': 'scheduled_skanta@teuthology', 'priority': 99, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 2126, 'sha1': 'd75f23820806da622f561ee5ee7551015285993f', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'rados', 'suite_branch': 'wip-bharath9-testing-2024-09-13-1444-quincy', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_d75f23820806da622f561ee5ee7551015285993f/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': 'd75f23820806da622f561ee5ee7551015285993f', 'targets': {'smithi110.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM=', 'smithi177.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_system_packages': {'deb': ['sqlite3'], 'rpm': ['sqlite-devel']}, 'flavor': 'default', 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}}, {'ceph': None}, {'exec': {'client.0': ['ceph osd pool create cephsqlite', "ceph auth get-or-create client.libcephsqlite mon 'profile simple-rados-client-with-blocklist' osd 'allow rwx pool=cephsqlite' >> /etc/ceph/ceph.keyring"]}}, {'exec': {'client.0': ['ceph_test_libcephsqlite --id libcephsqlite --no-log-to-stderr']}}, {'workunit': {'clients': {'client.0': ['rados/test_libcephsqlite.sh cephsqlite']}, 'env': {'CEPH_ARGS': '--id libcephsqlite --no-log-to-stderr'}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-15_00:54:39', 'tube': 'smithi', 'user': 'skanta', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-15T03:35:52.877 DEBUG:teuthology.orchestra.run.smithi110:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-15T03:35:52.879 DEBUG:teuthology.orchestra.run.smithi177:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-15T03:35:52.885 DEBUG:teuthology.orchestra.run.smithi110:> sudo install -d -m0777 -- /var/run/ceph 2024-09-15T03:35:52.887 DEBUG:teuthology.orchestra.run.smithi177:> sudo install -d -m0777 -- /var/run/ceph 2024-09-15T03:35:52.902 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-15T03:35:52.903 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/scratch_devs of=/dev/stdout 2024-09-15T03:35:52.948 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-15T03:35:52.948 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/vg_nvme/lv_1 2024-09-15T03:35:52.956 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-15T03:35:52.956 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T03:35:52.956 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 6h/6d Inode: 361 Links: 1 2024-09-15T03:35:52.957 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T03:35:52.957 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2024-09-15 03:32:23.656547076 +0000 2024-09-15T03:35:52.957 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2024-09-15 03:32:23.436552150 +0000 2024-09-15T03:35:52.957 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2024-09-15 03:32:23.436552150 +0000 2024-09-15T03:35:52.957 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: - 2024-09-15T03:35:52.957 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-15T03:35:52.972 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-09-15T03:35:52.972 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-09-15T03:35:52.972 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 0.0003058 s, 1.7 MB/s 2024-09-15T03:35:52.973 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-15T03:35:53.022 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/vg_nvme/lv_2 2024-09-15T03:35:53.029 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-15T03:35:53.029 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T03:35:53.029 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 6h/6d Inode: 364 Links: 1 2024-09-15T03:35:53.029 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T03:35:53.029 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2024-09-15 03:32:24.064537664 +0000 2024-09-15T03:35:53.029 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2024-09-15 03:32:23.844542739 +0000 2024-09-15T03:35:53.029 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2024-09-15 03:32:23.844542739 +0000 2024-09-15T03:35:53.029 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: - 2024-09-15T03:35:53.030 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-15T03:35:53.043 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-09-15T03:35:53.043 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-09-15T03:35:53.043 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 0.000297943 s, 1.7 MB/s 2024-09-15T03:35:53.044 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-15T03:35:53.098 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/vg_nvme/lv_3 2024-09-15T03:35:53.105 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-15T03:35:53.106 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T03:35:53.106 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 6h/6d Inode: 371 Links: 1 2024-09-15T03:35:53.106 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T03:35:53.106 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2024-09-15 03:32:24.496527699 +0000 2024-09-15T03:35:53.106 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2024-09-15 03:32:24.284532589 +0000 2024-09-15T03:35:53.106 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2024-09-15 03:32:24.284532589 +0000 2024-09-15T03:35:53.106 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: - 2024-09-15T03:35:53.106 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-15T03:35:53.118 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-09-15T03:35:53.118 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-09-15T03:35:53.118 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 0.000265931 s, 1.9 MB/s 2024-09-15T03:35:53.119 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-15T03:35:53.170 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/vg_nvme/lv_4 2024-09-15T03:35:53.176 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-15T03:35:53.176 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T03:35:53.176 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 6h/6d Inode: 380 Links: 1 2024-09-15T03:35:53.176 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T03:35:53.177 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2024-09-15 03:32:24.928517735 +0000 2024-09-15T03:35:53.177 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2024-09-15 03:32:24.700522994 +0000 2024-09-15T03:35:53.177 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2024-09-15 03:32:24.700522994 +0000 2024-09-15T03:35:53.177 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: - 2024-09-15T03:35:53.177 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-15T03:35:53.189 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-09-15T03:35:53.189 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-09-15T03:35:53.189 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 0.000289543 s, 1.8 MB/s 2024-09-15T03:35:53.190 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-15T03:35:53.242 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-15T03:35:53.242 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-15T03:35:53.242 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/scratch_devs of=/dev/stdout 2024-09-15T03:35:53.249 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-15T03:35:53.249 DEBUG:teuthology.orchestra.run.smithi177:> stat /dev/vg_nvme/lv_1 2024-09-15T03:35:53.256 INFO:teuthology.orchestra.run.smithi177.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-15T03:35:53.256 INFO:teuthology.orchestra.run.smithi177.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T03:35:53.256 INFO:teuthology.orchestra.run.smithi177.stdout:Device: 6h/6d Inode: 359 Links: 1 2024-09-15T03:35:53.256 INFO:teuthology.orchestra.run.smithi177.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T03:35:53.257 INFO:teuthology.orchestra.run.smithi177.stdout:Access: 2024-09-15 03:32:23.629878550 +0000 2024-09-15T03:35:53.257 INFO:teuthology.orchestra.run.smithi177.stdout:Modify: 2024-09-15 03:32:23.425883252 +0000 2024-09-15T03:35:53.257 INFO:teuthology.orchestra.run.smithi177.stdout:Change: 2024-09-15 03:32:23.425883252 +0000 2024-09-15T03:35:53.257 INFO:teuthology.orchestra.run.smithi177.stdout: Birth: - 2024-09-15T03:35:53.257 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-15T03:35:53.269 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records in 2024-09-15T03:35:53.269 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records out 2024-09-15T03:35:53.269 INFO:teuthology.orchestra.run.smithi177.stderr:512 bytes copied, 0.000431858 s, 1.2 MB/s 2024-09-15T03:35:53.270 DEBUG:teuthology.orchestra.run.smithi177:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-15T03:35:53.278 DEBUG:teuthology.orchestra.run.smithi177:> stat /dev/vg_nvme/lv_2 2024-09-15T03:35:53.283 INFO:teuthology.orchestra.run.smithi177.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-15T03:35:53.284 INFO:teuthology.orchestra.run.smithi177.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T03:35:53.284 INFO:teuthology.orchestra.run.smithi177.stdout:Device: 6h/6d Inode: 368 Links: 1 2024-09-15T03:35:53.284 INFO:teuthology.orchestra.run.smithi177.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T03:35:53.284 INFO:teuthology.orchestra.run.smithi177.stdout:Access: 2024-09-15 03:32:24.073868316 +0000 2024-09-15T03:35:53.284 INFO:teuthology.orchestra.run.smithi177.stdout:Modify: 2024-09-15 03:32:23.877872833 +0000 2024-09-15T03:35:53.284 INFO:teuthology.orchestra.run.smithi177.stdout:Change: 2024-09-15 03:32:23.877872833 +0000 2024-09-15T03:35:53.284 INFO:teuthology.orchestra.run.smithi177.stdout: Birth: - 2024-09-15T03:35:53.284 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-15T03:35:53.297 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records in 2024-09-15T03:35:53.297 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records out 2024-09-15T03:35:53.297 INFO:teuthology.orchestra.run.smithi177.stderr:512 bytes copied, 0.000418382 s, 1.2 MB/s 2024-09-15T03:35:53.299 DEBUG:teuthology.orchestra.run.smithi177:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-15T03:35:53.307 DEBUG:teuthology.orchestra.run.smithi177:> stat /dev/vg_nvme/lv_3 2024-09-15T03:35:53.361 INFO:teuthology.orchestra.run.smithi177.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-15T03:35:53.362 INFO:teuthology.orchestra.run.smithi177.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T03:35:53.362 INFO:teuthology.orchestra.run.smithi177.stdout:Device: 6h/6d Inode: 373 Links: 1 2024-09-15T03:35:53.362 INFO:teuthology.orchestra.run.smithi177.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T03:35:53.362 INFO:teuthology.orchestra.run.smithi177.stdout:Access: 2024-09-15 03:32:24.525857897 +0000 2024-09-15T03:35:53.362 INFO:teuthology.orchestra.run.smithi177.stdout:Modify: 2024-09-15 03:32:24.285863429 +0000 2024-09-15T03:35:53.362 INFO:teuthology.orchestra.run.smithi177.stdout:Change: 2024-09-15 03:32:24.285863429 +0000 2024-09-15T03:35:53.362 INFO:teuthology.orchestra.run.smithi177.stdout: Birth: - 2024-09-15T03:35:53.362 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-15T03:35:53.374 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records in 2024-09-15T03:35:53.374 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records out 2024-09-15T03:35:53.374 INFO:teuthology.orchestra.run.smithi177.stderr:512 bytes copied, 0.000403041 s, 1.3 MB/s 2024-09-15T03:35:53.375 DEBUG:teuthology.orchestra.run.smithi177:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-15T03:35:53.383 DEBUG:teuthology.orchestra.run.smithi177:> stat /dev/vg_nvme/lv_4 2024-09-15T03:35:53.389 INFO:teuthology.orchestra.run.smithi177.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-15T03:35:53.389 INFO:teuthology.orchestra.run.smithi177.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T03:35:53.389 INFO:teuthology.orchestra.run.smithi177.stdout:Device: 6h/6d Inode: 378 Links: 1 2024-09-15T03:35:53.389 INFO:teuthology.orchestra.run.smithi177.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T03:35:53.389 INFO:teuthology.orchestra.run.smithi177.stdout:Access: 2024-09-15 03:32:24.957847940 +0000 2024-09-15T03:35:53.390 INFO:teuthology.orchestra.run.smithi177.stdout:Modify: 2024-09-15 03:32:24.749852735 +0000 2024-09-15T03:35:53.390 INFO:teuthology.orchestra.run.smithi177.stdout:Change: 2024-09-15 03:32:24.749852735 +0000 2024-09-15T03:35:53.390 INFO:teuthology.orchestra.run.smithi177.stdout: Birth: - 2024-09-15T03:35:53.390 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-15T03:35:53.402 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records in 2024-09-15T03:35:53.402 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records out 2024-09-15T03:35:53.402 INFO:teuthology.orchestra.run.smithi177.stderr:512 bytes copied, 0.000363861 s, 1.4 MB/s 2024-09-15T03:35:53.403 DEBUG:teuthology.orchestra.run.smithi177:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-15T03:35:53.455 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-15T03:35:53.456 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi110.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi177.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-15T03:35:53.456 INFO:tasks.ceph:Generating config... 2024-09-15T03:35:53.457 INFO:tasks.ceph:[client] debug cephsqlite = 20 2024-09-15T03:35:53.457 INFO:tasks.ceph:[client] debug client = 20 2024-09-15T03:35:53.457 INFO:tasks.ceph:[client] debug ms = 1 2024-09-15T03:35:53.457 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-15T03:35:53.457 INFO:tasks.ceph:[global] mon election default strategy = 1 2024-09-15T03:35:53.457 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-15T03:35:53.458 INFO:tasks.ceph:[global] ms type = async 2024-09-15T03:35:53.458 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-15T03:35:53.458 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-15T03:35:53.458 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-15T03:35:53.458 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-15T03:35:53.458 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-15T03:35:53.458 INFO:tasks.ceph:[mon] mon min osdmap epochs = 50 2024-09-15T03:35:53.458 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2024-09-15T03:35:53.458 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2024-09-15T03:35:53.458 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2024-09-15T03:35:53.458 INFO:tasks.ceph:[mon] mon scrub interval = 300 2024-09-15T03:35:53.458 INFO:tasks.ceph:[mon] paxos service trim min = 10 2024-09-15T03:35:53.459 INFO:tasks.ceph:[osd] bluefs allocator = hybrid 2024-09-15T03:35:53.459 INFO:tasks.ceph:[osd] bluestore allocator = hybrid 2024-09-15T03:35:53.459 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-15T03:35:53.459 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-15T03:35:53.459 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2024-09-15T03:35:53.459 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-09-15T03:35:53.459 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-09-15T03:35:53.459 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-15T03:35:53.459 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-15T03:35:53.459 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-09-15T03:35:53.459 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-15T03:35:53.459 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-15T03:35:53.460 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-15T03:35:53.460 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2024-09-15T03:35:53.460 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2024-09-15T03:35:53.460 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-15T03:35:53.460 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2024-09-15T03:35:53.460 INFO:tasks.ceph:[osd] osd mclock profile = high_recovery_ops 2024-09-15T03:35:53.460 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-15T03:35:53.460 INFO:tasks.ceph:[osd] osd op queue = debug_random 2024-09-15T03:35:53.460 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2024-09-15T03:35:53.460 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-09-15T03:35:53.460 INFO:tasks.ceph:Setting up mon.a... 2024-09-15T03:35:53.461 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-15T03:35:53.497 INFO:teuthology.orchestra.run.smithi110.stdout:creating /etc/ceph/ceph.keyring 2024-09-15T03:35:53.500 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-15T03:35:53.536 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-15T03:35:53.546 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.110'), ('mon.c', '[v2:172.21.15.110:3301,v1:172.21.15.110:6790]'), ('mon.b', '172.21.15.177')] 2024-09-15T03:35:53.547 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.110,[v2:172.21.15.110:3301,v1:172.21.15.110:6790],172.21.15.177', 'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject socket failures': 5000, 'ms type': 'async'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug cephsqlite': 20, 'debug client': 20, 'debug ms': 1}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-09-15T03:35:53.547 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-15T03:35:53.547 DEBUG:teuthology.orchestra.run.smithi110:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-15T03:35:53.596 DEBUG:teuthology.orchestra.run.smithi110:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.110 --addv c '[v2:172.21.15.110:3301,v1:172.21.15.110:6790]' --add b 172.21.15.177 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-15T03:35:53.636 INFO:teuthology.orchestra.run.smithi110.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-15T03:35:53.636 INFO:teuthology.orchestra.run.smithi110.stdout:monmaptool: generated fsid d7bdbc6e-e9f0-40e5-921a-f7c68fa2d8c1 2024-09-15T03:35:53.636 INFO:teuthology.orchestra.run.smithi110.stdout:setting min_mon_release = octopus 2024-09-15T03:35:53.636 INFO:teuthology.orchestra.run.smithi110.stdout:epoch 0 2024-09-15T03:35:53.636 INFO:teuthology.orchestra.run.smithi110.stdout:fsid d7bdbc6e-e9f0-40e5-921a-f7c68fa2d8c1 2024-09-15T03:35:53.636 INFO:teuthology.orchestra.run.smithi110.stdout:last_changed 2024-09-15T03:35:53.634886+0000 2024-09-15T03:35:53.636 INFO:teuthology.orchestra.run.smithi110.stdout:created 2024-09-15T03:35:53.634886+0000 2024-09-15T03:35:53.636 INFO:teuthology.orchestra.run.smithi110.stdout:min_mon_release 15 (octopus) 2024-09-15T03:35:53.637 INFO:teuthology.orchestra.run.smithi110.stdout:election_strategy: 1 2024-09-15T03:35:53.637 INFO:teuthology.orchestra.run.smithi110.stdout:0: [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] mon.a 2024-09-15T03:35:53.637 INFO:teuthology.orchestra.run.smithi110.stdout:1: [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] mon.b 2024-09-15T03:35:53.637 INFO:teuthology.orchestra.run.smithi110.stdout:2: [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] mon.c 2024-09-15T03:35:53.637 INFO:teuthology.orchestra.run.smithi110.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-15T03:35:53.638 DEBUG:teuthology.orchestra.run.smithi110:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-15T03:35:53.642 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID d7bdbc6e-e9f0-40e5-921a-f7c68fa2d8c1... 2024-09-15T03:35:53.643 DEBUG:teuthology.orchestra.run.smithi110:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-15T03:35:53.691 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-15T03:35:53.725 INFO:teuthology.orchestra.run.smithi110.stdout:[global] 2024-09-15T03:35:53.725 INFO:teuthology.orchestra.run.smithi110.stdout: chdir = "" 2024-09-15T03:35:53.725 INFO:teuthology.orchestra.run.smithi110.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-15T03:35:53.725 INFO:teuthology.orchestra.run.smithi110.stdout: auth supported = cephx 2024-09-15T03:35:53.725 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-15T03:35:53.725 INFO:teuthology.orchestra.run.smithi110.stdout: filestore xattr use omap = true 2024-09-15T03:35:53.725 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-15T03:35:53.725 INFO:teuthology.orchestra.run.smithi110.stdout: mon clock drift allowed = 1.000 2024-09-15T03:35:53.726 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-15T03:35:53.726 INFO:teuthology.orchestra.run.smithi110.stdout: osd crush chooseleaf type = 0 2024-09-15T03:35:53.726 INFO:teuthology.orchestra.run.smithi110.stdout: auth debug = true 2024-09-15T03:35:53.726 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-15T03:35:53.726 INFO:teuthology.orchestra.run.smithi110.stdout: ms die on old message = true 2024-09-15T03:35:53.726 INFO:teuthology.orchestra.run.smithi110.stdout: ms die on bug = true 2024-09-15T03:35:53.726 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-15T03:35:53.726 INFO:teuthology.orchestra.run.smithi110.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-15T03:35:53.726 INFO:teuthology.orchestra.run.smithi110.stdout: mon pg warn max object skew = 0 2024-09-15T03:35:53.726 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-15T03:35:53.726 INFO:teuthology.orchestra.run.smithi110.stdout: # disable pg_autoscaler by default for new pools 2024-09-15T03:35:53.726 INFO:teuthology.orchestra.run.smithi110.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-15T03:35:53.727 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-15T03:35:53.727 INFO:teuthology.orchestra.run.smithi110.stdout: osd pool default size = 2 2024-09-15T03:35:53.727 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-15T03:35:53.727 INFO:teuthology.orchestra.run.smithi110.stdout: mon osd allow primary affinity = true 2024-09-15T03:35:53.727 INFO:teuthology.orchestra.run.smithi110.stdout: mon osd allow pg remap = true 2024-09-15T03:35:53.727 INFO:teuthology.orchestra.run.smithi110.stdout: mon warn on legacy crush tunables = false 2024-09-15T03:35:53.727 INFO:teuthology.orchestra.run.smithi110.stdout: mon warn on crush straw calc version zero = false 2024-09-15T03:35:53.727 INFO:teuthology.orchestra.run.smithi110.stdout: mon warn on no sortbitwise = false 2024-09-15T03:35:53.727 INFO:teuthology.orchestra.run.smithi110.stdout: mon warn on osd down out interval zero = false 2024-09-15T03:35:53.727 INFO:teuthology.orchestra.run.smithi110.stdout: mon warn on too few osds = false 2024-09-15T03:35:53.727 INFO:teuthology.orchestra.run.smithi110.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-15T03:35:53.727 INFO:teuthology.orchestra.run.smithi110.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-15T03:35:53.727 INFO:teuthology.orchestra.run.smithi110.stdout: mon_allow_pool_size_one = true 2024-09-15T03:35:53.727 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-15T03:35:53.728 INFO:teuthology.orchestra.run.smithi110.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-15T03:35:53.728 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-15T03:35:53.728 INFO:teuthology.orchestra.run.smithi110.stdout: osd default data pool replay window = 5 2024-09-15T03:35:53.728 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-15T03:35:53.728 INFO:teuthology.orchestra.run.smithi110.stdout: mon allow pool delete = true 2024-09-15T03:35:53.728 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-15T03:35:53.728 INFO:teuthology.orchestra.run.smithi110.stdout: mon cluster log file level = debug 2024-09-15T03:35:53.728 INFO:teuthology.orchestra.run.smithi110.stdout: debug asserts on shutdown = true 2024-09-15T03:35:53.728 INFO:teuthology.orchestra.run.smithi110.stdout: mon health detail to clog = false 2024-09-15T03:35:53.728 INFO:teuthology.orchestra.run.smithi110.stdout: mon host = "172.21.15.110,[v2:172.21.15.110:3301,v1:172.21.15.110:6790],172.21.15.177" 2024-09-15T03:35:53.728 INFO:teuthology.orchestra.run.smithi110.stdout: mon client directed command retry = 5 2024-09-15T03:35:53.728 INFO:teuthology.orchestra.run.smithi110.stdout: mon election default strategy = 1 2024-09-15T03:35:53.728 INFO:teuthology.orchestra.run.smithi110.stdout: ms inject socket failures = 5000 2024-09-15T03:35:53.729 INFO:teuthology.orchestra.run.smithi110.stdout: ms type = async 2024-09-15T03:35:53.729 INFO:teuthology.orchestra.run.smithi110.stdout: fsid = d7bdbc6e-e9f0-40e5-921a-f7c68fa2d8c1 2024-09-15T03:35:53.729 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-15T03:35:53.729 INFO:teuthology.orchestra.run.smithi110.stdout:[osd] 2024-09-15T03:35:53.729 INFO:teuthology.orchestra.run.smithi110.stdout: osd journal size = 100 2024-09-15T03:35:53.729 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-15T03:35:53.729 INFO:teuthology.orchestra.run.smithi110.stdout: osd scrub load threshold = 5.0 2024-09-15T03:35:53.729 INFO:teuthology.orchestra.run.smithi110.stdout: osd scrub max interval = 600 2024-09-15T03:35:53.729 INFO:teuthology.orchestra.run.smithi110.stdout: osd mclock profile = high_recovery_ops 2024-09-15T03:35:53.729 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-15T03:35:53.729 INFO:teuthology.orchestra.run.smithi110.stdout: osd recover clone overlap = true 2024-09-15T03:35:53.729 INFO:teuthology.orchestra.run.smithi110.stdout: osd recovery max chunk = 1048576 2024-09-15T03:35:53.729 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-15T03:35:53.729 INFO:teuthology.orchestra.run.smithi110.stdout: osd debug shutdown = true 2024-09-15T03:35:53.730 INFO:teuthology.orchestra.run.smithi110.stdout: osd debug op order = true 2024-09-15T03:35:53.730 INFO:teuthology.orchestra.run.smithi110.stdout: osd debug verify stray on activate = true 2024-09-15T03:35:53.730 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-15T03:35:53.730 INFO:teuthology.orchestra.run.smithi110.stdout: osd open classes on start = true 2024-09-15T03:35:53.730 INFO:teuthology.orchestra.run.smithi110.stdout: osd debug pg log writeout = true 2024-09-15T03:35:53.730 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-15T03:35:53.730 INFO:teuthology.orchestra.run.smithi110.stdout: osd deep scrub update digest min age = 30 2024-09-15T03:35:53.730 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-15T03:35:53.730 INFO:teuthology.orchestra.run.smithi110.stdout: osd map max advance = 10 2024-09-15T03:35:53.730 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-15T03:35:53.730 INFO:teuthology.orchestra.run.smithi110.stdout: journal zero on create = true 2024-09-15T03:35:53.730 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-15T03:35:53.730 INFO:teuthology.orchestra.run.smithi110.stdout: filestore ondisk finisher threads = 3 2024-09-15T03:35:53.730 INFO:teuthology.orchestra.run.smithi110.stdout: filestore apply finisher threads = 3 2024-09-15T03:35:53.731 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-15T03:35:53.731 INFO:teuthology.orchestra.run.smithi110.stdout: bdev debug aio = true 2024-09-15T03:35:53.731 INFO:teuthology.orchestra.run.smithi110.stdout: osd debug misdirected ops = true 2024-09-15T03:35:53.731 INFO:teuthology.orchestra.run.smithi110.stdout: bluefs allocator = hybrid 2024-09-15T03:35:53.731 INFO:teuthology.orchestra.run.smithi110.stdout: bluestore allocator = hybrid 2024-09-15T03:35:53.731 INFO:teuthology.orchestra.run.smithi110.stdout: bluestore block size = 96636764160 2024-09-15T03:35:53.731 INFO:teuthology.orchestra.run.smithi110.stdout: bluestore fsck on mount = True 2024-09-15T03:35:53.731 INFO:teuthology.orchestra.run.smithi110.stdout: bluestore zero block detection = True 2024-09-15T03:35:53.731 INFO:teuthology.orchestra.run.smithi110.stdout: debug bluefs = 20 2024-09-15T03:35:53.731 INFO:teuthology.orchestra.run.smithi110.stdout: debug bluestore = 20 2024-09-15T03:35:53.731 INFO:teuthology.orchestra.run.smithi110.stdout: debug ms = 1 2024-09-15T03:35:53.731 INFO:teuthology.orchestra.run.smithi110.stdout: debug osd = 20 2024-09-15T03:35:53.731 INFO:teuthology.orchestra.run.smithi110.stdout: debug rocksdb = 10 2024-09-15T03:35:53.732 INFO:teuthology.orchestra.run.smithi110.stdout: mon osd backfillfull_ratio = 0.85 2024-09-15T03:35:53.732 INFO:teuthology.orchestra.run.smithi110.stdout: mon osd full ratio = 0.9 2024-09-15T03:35:53.732 INFO:teuthology.orchestra.run.smithi110.stdout: mon osd nearfull ratio = 0.8 2024-09-15T03:35:53.732 INFO:teuthology.orchestra.run.smithi110.stdout: osd debug verify cached snaps = True 2024-09-15T03:35:53.732 INFO:teuthology.orchestra.run.smithi110.stdout: osd debug verify missing on start = True 2024-09-15T03:35:53.732 INFO:teuthology.orchestra.run.smithi110.stdout: osd failsafe full ratio = 0.95 2024-09-15T03:35:53.732 INFO:teuthology.orchestra.run.smithi110.stdout: osd mclock override recovery settings = True 2024-09-15T03:35:53.732 INFO:teuthology.orchestra.run.smithi110.stdout: osd objectstore = bluestore 2024-09-15T03:35:53.732 INFO:teuthology.orchestra.run.smithi110.stdout: osd op queue = debug_random 2024-09-15T03:35:53.732 INFO:teuthology.orchestra.run.smithi110.stdout: osd op queue cut off = debug_random 2024-09-15T03:35:53.732 INFO:teuthology.orchestra.run.smithi110.stdout: osd shutdown pgref assert = True 2024-09-15T03:35:53.732 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-15T03:35:53.732 INFO:teuthology.orchestra.run.smithi110.stdout:[mgr] 2024-09-15T03:35:53.733 INFO:teuthology.orchestra.run.smithi110.stdout: debug ms = 1 2024-09-15T03:35:53.733 INFO:teuthology.orchestra.run.smithi110.stdout: debug mgr = 20 2024-09-15T03:35:53.733 INFO:teuthology.orchestra.run.smithi110.stdout: debug mon = 20 2024-09-15T03:35:53.733 INFO:teuthology.orchestra.run.smithi110.stdout: debug auth = 20 2024-09-15T03:35:53.733 INFO:teuthology.orchestra.run.smithi110.stdout: mon reweight min pgs per osd = 4 2024-09-15T03:35:53.733 INFO:teuthology.orchestra.run.smithi110.stdout: mon reweight min bytes per osd = 10 2024-09-15T03:35:53.733 INFO:teuthology.orchestra.run.smithi110.stdout: mgr/telemetry/nag = false 2024-09-15T03:35:53.733 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-15T03:35:53.733 INFO:teuthology.orchestra.run.smithi110.stdout:[mon] 2024-09-15T03:35:53.733 INFO:teuthology.orchestra.run.smithi110.stdout: debug ms = 1 2024-09-15T03:35:53.733 INFO:teuthology.orchestra.run.smithi110.stdout: debug mon = 20 2024-09-15T03:35:53.733 INFO:teuthology.orchestra.run.smithi110.stdout: debug paxos = 20 2024-09-15T03:35:53.733 INFO:teuthology.orchestra.run.smithi110.stdout: debug auth = 20 2024-09-15T03:35:53.734 INFO:teuthology.orchestra.run.smithi110.stdout: mon data avail warn = 5 2024-09-15T03:35:53.734 INFO:teuthology.orchestra.run.smithi110.stdout: mon mgr mkfs grace = 240 2024-09-15T03:35:53.734 INFO:teuthology.orchestra.run.smithi110.stdout: mon reweight min pgs per osd = 4 2024-09-15T03:35:53.734 INFO:teuthology.orchestra.run.smithi110.stdout: mon osd reporter subtree level = osd 2024-09-15T03:35:53.734 INFO:teuthology.orchestra.run.smithi110.stdout: mon osd prime pg temp = true 2024-09-15T03:35:53.734 INFO:teuthology.orchestra.run.smithi110.stdout: mon reweight min bytes per osd = 10 2024-09-15T03:35:53.734 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-15T03:35:53.734 INFO:teuthology.orchestra.run.smithi110.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-15T03:35:53.734 INFO:teuthology.orchestra.run.smithi110.stdout: auth mon ticket ttl = 660 # 11m 2024-09-15T03:35:53.734 INFO:teuthology.orchestra.run.smithi110.stdout: auth service ticket ttl = 240 # 4m 2024-09-15T03:35:53.734 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-15T03:35:53.734 INFO:teuthology.orchestra.run.smithi110.stdout: # don't complain about insecure global_id in the test suite 2024-09-15T03:35:53.734 INFO:teuthology.orchestra.run.smithi110.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-15T03:35:53.735 INFO:teuthology.orchestra.run.smithi110.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-15T03:35:53.735 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-15T03:35:53.735 INFO:teuthology.orchestra.run.smithi110.stdout: # 1m isn't quite enough 2024-09-15T03:35:53.735 INFO:teuthology.orchestra.run.smithi110.stdout: mon_down_mkfs_grace = 2m 2024-09-15T03:35:53.735 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-15T03:35:53.735 INFO:teuthology.orchestra.run.smithi110.stdout: mon_warn_on_filestore_osds = false 2024-09-15T03:35:53.735 INFO:teuthology.orchestra.run.smithi110.stdout: mon min osdmap epochs = 50 2024-09-15T03:35:53.735 INFO:teuthology.orchestra.run.smithi110.stdout: mon osdmap full prune interval = 2 2024-09-15T03:35:53.735 INFO:teuthology.orchestra.run.smithi110.stdout: mon osdmap full prune min = 15 2024-09-15T03:35:53.735 INFO:teuthology.orchestra.run.smithi110.stdout: mon osdmap full prune txsize = 2 2024-09-15T03:35:53.735 INFO:teuthology.orchestra.run.smithi110.stdout: mon scrub interval = 300 2024-09-15T03:35:53.735 INFO:teuthology.orchestra.run.smithi110.stdout: paxos service trim min = 10 2024-09-15T03:35:53.735 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-15T03:35:53.736 INFO:teuthology.orchestra.run.smithi110.stdout:[client] 2024-09-15T03:35:53.736 INFO:teuthology.orchestra.run.smithi110.stdout: rgw cache enabled = true 2024-09-15T03:35:53.736 INFO:teuthology.orchestra.run.smithi110.stdout: rgw enable ops log = true 2024-09-15T03:35:53.736 INFO:teuthology.orchestra.run.smithi110.stdout: rgw enable usage log = true 2024-09-15T03:35:53.736 INFO:teuthology.orchestra.run.smithi110.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-15T03:35:53.736 INFO:teuthology.orchestra.run.smithi110.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-15T03:35:53.736 INFO:teuthology.orchestra.run.smithi110.stdout: debug cephsqlite = 20 2024-09-15T03:35:53.736 INFO:teuthology.orchestra.run.smithi110.stdout: debug client = 20 2024-09-15T03:35:53.736 INFO:teuthology.orchestra.run.smithi110.stdout: debug ms = 1 2024-09-15T03:35:53.736 INFO:teuthology.orchestra.run.smithi110.stdout:[mon.a] 2024-09-15T03:35:53.736 INFO:teuthology.orchestra.run.smithi110.stdout:[mon.c] 2024-09-15T03:35:53.736 INFO:teuthology.orchestra.run.smithi110.stdout:[mon.b] 2024-09-15T03:35:53.738 INFO:teuthology.orchestra.run.smithi177.stdout:[global] 2024-09-15T03:35:53.738 INFO:teuthology.orchestra.run.smithi177.stdout: chdir = "" 2024-09-15T03:35:53.738 INFO:teuthology.orchestra.run.smithi177.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-15T03:35:53.738 INFO:teuthology.orchestra.run.smithi177.stdout: auth supported = cephx 2024-09-15T03:35:53.738 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-15T03:35:53.738 INFO:teuthology.orchestra.run.smithi177.stdout: filestore xattr use omap = true 2024-09-15T03:35:53.738 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-15T03:35:53.738 INFO:teuthology.orchestra.run.smithi177.stdout: mon clock drift allowed = 1.000 2024-09-15T03:35:53.738 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-15T03:35:53.738 INFO:teuthology.orchestra.run.smithi177.stdout: osd crush chooseleaf type = 0 2024-09-15T03:35:53.739 INFO:teuthology.orchestra.run.smithi177.stdout: auth debug = true 2024-09-15T03:35:53.739 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-15T03:35:53.739 INFO:teuthology.orchestra.run.smithi177.stdout: ms die on old message = true 2024-09-15T03:35:53.739 INFO:teuthology.orchestra.run.smithi177.stdout: ms die on bug = true 2024-09-15T03:35:53.739 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-15T03:35:53.739 INFO:teuthology.orchestra.run.smithi177.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-15T03:35:53.739 INFO:teuthology.orchestra.run.smithi177.stdout: mon pg warn max object skew = 0 2024-09-15T03:35:53.739 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-15T03:35:53.739 INFO:teuthology.orchestra.run.smithi177.stdout: # disable pg_autoscaler by default for new pools 2024-09-15T03:35:53.739 INFO:teuthology.orchestra.run.smithi177.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-15T03:35:53.739 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-15T03:35:53.739 INFO:teuthology.orchestra.run.smithi177.stdout: osd pool default size = 2 2024-09-15T03:35:53.740 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-15T03:35:53.740 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd allow primary affinity = true 2024-09-15T03:35:53.740 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd allow pg remap = true 2024-09-15T03:35:53.740 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on legacy crush tunables = false 2024-09-15T03:35:53.740 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on crush straw calc version zero = false 2024-09-15T03:35:53.740 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on no sortbitwise = false 2024-09-15T03:35:53.740 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on osd down out interval zero = false 2024-09-15T03:35:53.740 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on too few osds = false 2024-09-15T03:35:53.740 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-15T03:35:53.740 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-15T03:35:53.740 INFO:teuthology.orchestra.run.smithi177.stdout: mon_allow_pool_size_one = true 2024-09-15T03:35:53.740 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-15T03:35:53.741 INFO:teuthology.orchestra.run.smithi177.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-15T03:35:53.741 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-15T03:35:53.741 INFO:teuthology.orchestra.run.smithi177.stdout: osd default data pool replay window = 5 2024-09-15T03:35:53.741 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-15T03:35:53.741 INFO:teuthology.orchestra.run.smithi177.stdout: mon allow pool delete = true 2024-09-15T03:35:53.741 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-15T03:35:53.741 INFO:teuthology.orchestra.run.smithi177.stdout: mon cluster log file level = debug 2024-09-15T03:35:53.741 INFO:teuthology.orchestra.run.smithi177.stdout: debug asserts on shutdown = true 2024-09-15T03:35:53.741 INFO:teuthology.orchestra.run.smithi177.stdout: mon health detail to clog = false 2024-09-15T03:35:53.741 INFO:teuthology.orchestra.run.smithi177.stdout: mon host = "172.21.15.110,[v2:172.21.15.110:3301,v1:172.21.15.110:6790],172.21.15.177" 2024-09-15T03:35:53.741 INFO:teuthology.orchestra.run.smithi177.stdout: mon client directed command retry = 5 2024-09-15T03:35:53.741 INFO:teuthology.orchestra.run.smithi177.stdout: mon election default strategy = 1 2024-09-15T03:35:53.741 INFO:teuthology.orchestra.run.smithi177.stdout: ms inject socket failures = 5000 2024-09-15T03:35:53.742 INFO:teuthology.orchestra.run.smithi177.stdout: ms type = async 2024-09-15T03:35:53.742 INFO:teuthology.orchestra.run.smithi177.stdout: fsid = d7bdbc6e-e9f0-40e5-921a-f7c68fa2d8c1 2024-09-15T03:35:53.742 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-15T03:35:53.742 INFO:teuthology.orchestra.run.smithi177.stdout:[osd] 2024-09-15T03:35:53.742 INFO:teuthology.orchestra.run.smithi177.stdout: osd journal size = 100 2024-09-15T03:35:53.742 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-15T03:35:53.742 INFO:teuthology.orchestra.run.smithi177.stdout: osd scrub load threshold = 5.0 2024-09-15T03:35:53.742 INFO:teuthology.orchestra.run.smithi177.stdout: osd scrub max interval = 600 2024-09-15T03:35:53.742 INFO:teuthology.orchestra.run.smithi177.stdout: osd mclock profile = high_recovery_ops 2024-09-15T03:35:53.742 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-15T03:35:53.742 INFO:teuthology.orchestra.run.smithi177.stdout: osd recover clone overlap = true 2024-09-15T03:35:53.742 INFO:teuthology.orchestra.run.smithi177.stdout: osd recovery max chunk = 1048576 2024-09-15T03:35:53.742 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-15T03:35:53.743 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug shutdown = true 2024-09-15T03:35:53.743 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug op order = true 2024-09-15T03:35:53.743 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug verify stray on activate = true 2024-09-15T03:35:53.743 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-15T03:35:53.743 INFO:teuthology.orchestra.run.smithi177.stdout: osd open classes on start = true 2024-09-15T03:35:53.743 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug pg log writeout = true 2024-09-15T03:35:53.743 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-15T03:35:53.743 INFO:teuthology.orchestra.run.smithi177.stdout: osd deep scrub update digest min age = 30 2024-09-15T03:35:53.743 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-15T03:35:53.743 INFO:teuthology.orchestra.run.smithi177.stdout: osd map max advance = 10 2024-09-15T03:35:53.743 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-15T03:35:53.743 INFO:teuthology.orchestra.run.smithi177.stdout: journal zero on create = true 2024-09-15T03:35:53.743 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-15T03:35:53.744 INFO:teuthology.orchestra.run.smithi177.stdout: filestore ondisk finisher threads = 3 2024-09-15T03:35:53.744 INFO:teuthology.orchestra.run.smithi177.stdout: filestore apply finisher threads = 3 2024-09-15T03:35:53.744 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-15T03:35:53.744 INFO:teuthology.orchestra.run.smithi177.stdout: bdev debug aio = true 2024-09-15T03:35:53.744 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug misdirected ops = true 2024-09-15T03:35:53.744 INFO:teuthology.orchestra.run.smithi177.stdout: bluefs allocator = hybrid 2024-09-15T03:35:53.744 INFO:teuthology.orchestra.run.smithi177.stdout: bluestore allocator = hybrid 2024-09-15T03:35:53.744 INFO:teuthology.orchestra.run.smithi177.stdout: bluestore block size = 96636764160 2024-09-15T03:35:53.744 INFO:teuthology.orchestra.run.smithi177.stdout: bluestore fsck on mount = True 2024-09-15T03:35:53.744 INFO:teuthology.orchestra.run.smithi177.stdout: bluestore zero block detection = True 2024-09-15T03:35:53.744 INFO:teuthology.orchestra.run.smithi177.stdout: debug bluefs = 20 2024-09-15T03:35:53.744 INFO:teuthology.orchestra.run.smithi177.stdout: debug bluestore = 20 2024-09-15T03:35:53.744 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2024-09-15T03:35:53.745 INFO:teuthology.orchestra.run.smithi177.stdout: debug osd = 20 2024-09-15T03:35:53.745 INFO:teuthology.orchestra.run.smithi177.stdout: debug rocksdb = 10 2024-09-15T03:35:53.745 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd backfillfull_ratio = 0.85 2024-09-15T03:35:53.745 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd full ratio = 0.9 2024-09-15T03:35:53.745 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd nearfull ratio = 0.8 2024-09-15T03:35:53.745 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug verify cached snaps = True 2024-09-15T03:35:53.745 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug verify missing on start = True 2024-09-15T03:35:53.745 INFO:teuthology.orchestra.run.smithi177.stdout: osd failsafe full ratio = 0.95 2024-09-15T03:35:53.745 INFO:teuthology.orchestra.run.smithi177.stdout: osd mclock override recovery settings = True 2024-09-15T03:35:53.745 INFO:teuthology.orchestra.run.smithi177.stdout: osd objectstore = bluestore 2024-09-15T03:35:53.745 INFO:teuthology.orchestra.run.smithi177.stdout: osd op queue = debug_random 2024-09-15T03:35:53.745 INFO:teuthology.orchestra.run.smithi177.stdout: osd op queue cut off = debug_random 2024-09-15T03:35:53.745 INFO:teuthology.orchestra.run.smithi177.stdout: osd shutdown pgref assert = True 2024-09-15T03:35:53.746 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-15T03:35:53.746 INFO:teuthology.orchestra.run.smithi177.stdout:[mgr] 2024-09-15T03:35:53.746 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2024-09-15T03:35:53.746 INFO:teuthology.orchestra.run.smithi177.stdout: debug mgr = 20 2024-09-15T03:35:53.746 INFO:teuthology.orchestra.run.smithi177.stdout: debug mon = 20 2024-09-15T03:35:53.746 INFO:teuthology.orchestra.run.smithi177.stdout: debug auth = 20 2024-09-15T03:35:53.746 INFO:teuthology.orchestra.run.smithi177.stdout: mon reweight min pgs per osd = 4 2024-09-15T03:35:53.746 INFO:teuthology.orchestra.run.smithi177.stdout: mon reweight min bytes per osd = 10 2024-09-15T03:35:53.746 INFO:teuthology.orchestra.run.smithi177.stdout: mgr/telemetry/nag = false 2024-09-15T03:35:53.746 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-15T03:35:53.746 INFO:teuthology.orchestra.run.smithi177.stdout:[mon] 2024-09-15T03:35:53.746 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2024-09-15T03:35:53.746 INFO:teuthology.orchestra.run.smithi177.stdout: debug mon = 20 2024-09-15T03:35:53.746 INFO:teuthology.orchestra.run.smithi177.stdout: debug paxos = 20 2024-09-15T03:35:53.747 INFO:teuthology.orchestra.run.smithi177.stdout: debug auth = 20 2024-09-15T03:35:53.747 INFO:teuthology.orchestra.run.smithi177.stdout: mon data avail warn = 5 2024-09-15T03:35:53.747 INFO:teuthology.orchestra.run.smithi177.stdout: mon mgr mkfs grace = 240 2024-09-15T03:35:53.747 INFO:teuthology.orchestra.run.smithi177.stdout: mon reweight min pgs per osd = 4 2024-09-15T03:35:53.747 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd reporter subtree level = osd 2024-09-15T03:35:53.747 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd prime pg temp = true 2024-09-15T03:35:53.747 INFO:teuthology.orchestra.run.smithi177.stdout: mon reweight min bytes per osd = 10 2024-09-15T03:35:53.747 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-15T03:35:53.747 INFO:teuthology.orchestra.run.smithi177.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-15T03:35:53.747 INFO:teuthology.orchestra.run.smithi177.stdout: auth mon ticket ttl = 660 # 11m 2024-09-15T03:35:53.747 INFO:teuthology.orchestra.run.smithi177.stdout: auth service ticket ttl = 240 # 4m 2024-09-15T03:35:53.747 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-15T03:35:53.748 INFO:teuthology.orchestra.run.smithi177.stdout: # don't complain about insecure global_id in the test suite 2024-09-15T03:35:53.748 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-15T03:35:53.748 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-15T03:35:53.748 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-15T03:35:53.748 INFO:teuthology.orchestra.run.smithi177.stdout: # 1m isn't quite enough 2024-09-15T03:35:53.748 INFO:teuthology.orchestra.run.smithi177.stdout: mon_down_mkfs_grace = 2m 2024-09-15T03:35:53.748 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-15T03:35:53.748 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_filestore_osds = false 2024-09-15T03:35:53.748 INFO:teuthology.orchestra.run.smithi177.stdout: mon min osdmap epochs = 50 2024-09-15T03:35:53.748 INFO:teuthology.orchestra.run.smithi177.stdout: mon osdmap full prune interval = 2 2024-09-15T03:35:53.749 INFO:teuthology.orchestra.run.smithi177.stdout: mon osdmap full prune min = 15 2024-09-15T03:35:53.749 INFO:teuthology.orchestra.run.smithi177.stdout: mon osdmap full prune txsize = 2 2024-09-15T03:35:53.749 INFO:teuthology.orchestra.run.smithi177.stdout: mon scrub interval = 300 2024-09-15T03:35:53.749 INFO:teuthology.orchestra.run.smithi177.stdout: paxos service trim min = 10 2024-09-15T03:35:53.749 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-15T03:35:53.749 INFO:teuthology.orchestra.run.smithi177.stdout:[client] 2024-09-15T03:35:53.749 INFO:teuthology.orchestra.run.smithi177.stdout: rgw cache enabled = true 2024-09-15T03:35:53.749 INFO:teuthology.orchestra.run.smithi177.stdout: rgw enable ops log = true 2024-09-15T03:35:53.749 INFO:teuthology.orchestra.run.smithi177.stdout: rgw enable usage log = true 2024-09-15T03:35:53.749 INFO:teuthology.orchestra.run.smithi177.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-15T03:35:53.749 INFO:teuthology.orchestra.run.smithi177.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-15T03:35:53.749 INFO:teuthology.orchestra.run.smithi177.stdout: debug cephsqlite = 20 2024-09-15T03:35:53.749 INFO:teuthology.orchestra.run.smithi177.stdout: debug client = 20 2024-09-15T03:35:53.749 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2024-09-15T03:35:53.750 INFO:teuthology.orchestra.run.smithi177.stdout:[mon.a] 2024-09-15T03:35:53.750 INFO:teuthology.orchestra.run.smithi177.stdout:[mon.c] 2024-09-15T03:35:53.750 INFO:teuthology.orchestra.run.smithi177.stdout:[mon.b] 2024-09-15T03:35:53.750 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-15T03:35:53.750 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-15T03:35:53.797 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-15T03:35:53.797 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-15T03:35:53.797 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-15T03:35:53.803 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-15T03:35:53.803 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-15T03:35:53.810 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi110.front.sepia.ceph.com 2024-09-15T03:35:53.810 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-15T03:35:53.810 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-15T03:35:53.810 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-15T03:35:53.833 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-15T03:35:53.833 DEBUG:teuthology.orchestra.run.smithi110:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-15T03:35:53.837 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi177.front.sepia.ceph.com 2024-09-15T03:35:53.838 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-15T03:35:53.838 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-15T03:35:53.838 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-15T03:35:53.867 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-15T03:35:53.867 DEBUG:teuthology.orchestra.run.smithi177:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-15T03:35:53.917 INFO:tasks.ceph:Setting up mon nodes... 2024-09-15T03:35:53.917 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-15T03:35:53.917 DEBUG:teuthology.orchestra.run.smithi110:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-15T03:35:53.963 INFO:teuthology.orchestra.run.smithi110.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-15T03:35:53.966 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-15T03:35:54.010 INFO:teuthology.orchestra.run.smithi177.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-15T03:35:54.012 INFO:tasks.ceph:Setting up mds nodes... 2024-09-15T03:35:54.012 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-15T03:35:54.013 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-15T03:35:54.050 INFO:teuthology.orchestra.run.smithi110.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-15T03:35:54.063 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-15T03:35:54.100 INFO:teuthology.orchestra.run.smithi177.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-15T03:35:54.113 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-15T03:35:54.114 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi110.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi177.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-15T03:35:54.114 DEBUG:teuthology.orchestra.run.smithi110:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-15T03:35:54.129 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-15T03:35:54.129 INFO:tasks.ceph:role: osd.0 2024-09-15T03:35:54.129 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi110.front.sepia.ceph.com 2024-09-15T03:35:54.129 DEBUG:teuthology.orchestra.run.smithi110:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-15T03:35:54.172 INFO:teuthology.orchestra.run.smithi110.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T03:35:54.172 INFO:teuthology.orchestra.run.smithi110.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T03:35:54.172 INFO:teuthology.orchestra.run.smithi110.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T03:35:54.172 INFO:teuthology.orchestra.run.smithi110.stdout: = reflink=1 2024-09-15T03:35:54.172 INFO:teuthology.orchestra.run.smithi110.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T03:35:54.172 INFO:teuthology.orchestra.run.smithi110.stdout: = sunit=0 swidth=0 blks 2024-09-15T03:35:54.172 INFO:teuthology.orchestra.run.smithi110.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T03:35:54.172 INFO:teuthology.orchestra.run.smithi110.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-15T03:35:54.172 INFO:teuthology.orchestra.run.smithi110.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T03:35:54.173 INFO:teuthology.orchestra.run.smithi110.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T03:35:54.173 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi110.front.sepia.ceph.com -o noatime 2024-09-15T03:35:54.174 DEBUG:teuthology.orchestra.run.smithi110:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-15T03:35:54.239 DEBUG:teuthology.orchestra.run.smithi110:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-15T03:35:54.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T03:35:54.249 INFO:teuthology.orchestra.run.smithi110.stderr:sudo: /sbin/restorecon: command not found 2024-09-15T03:35:54.249 DEBUG:teuthology.orchestra.run.smithi110:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-15T03:35:54.261 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-15T03:35:54.261 INFO:tasks.ceph:role: osd.1 2024-09-15T03:35:54.261 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi110.front.sepia.ceph.com 2024-09-15T03:35:54.261 DEBUG:teuthology.orchestra.run.smithi110:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-15T03:35:54.344 INFO:teuthology.orchestra.run.smithi110.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T03:35:54.344 INFO:teuthology.orchestra.run.smithi110.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T03:35:54.344 INFO:teuthology.orchestra.run.smithi110.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T03:35:54.344 INFO:teuthology.orchestra.run.smithi110.stdout: = reflink=1 2024-09-15T03:35:54.344 INFO:teuthology.orchestra.run.smithi110.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T03:35:54.344 INFO:teuthology.orchestra.run.smithi110.stdout: = sunit=0 swidth=0 blks 2024-09-15T03:35:54.344 INFO:teuthology.orchestra.run.smithi110.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T03:35:54.344 INFO:teuthology.orchestra.run.smithi110.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-15T03:35:54.345 INFO:teuthology.orchestra.run.smithi110.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T03:35:54.345 INFO:teuthology.orchestra.run.smithi110.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T03:35:54.345 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi110.front.sepia.ceph.com -o noatime 2024-09-15T03:35:54.346 DEBUG:teuthology.orchestra.run.smithi110:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-15T03:35:54.365 DEBUG:teuthology.orchestra.run.smithi110:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-15T03:35:54.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T03:35:54.373 INFO:teuthology.orchestra.run.smithi110.stderr:sudo: /sbin/restorecon: command not found 2024-09-15T03:35:54.373 DEBUG:teuthology.orchestra.run.smithi110:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-15T03:35:54.431 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-15T03:35:54.431 INFO:tasks.ceph:role: osd.2 2024-09-15T03:35:54.431 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi110.front.sepia.ceph.com 2024-09-15T03:35:54.432 DEBUG:teuthology.orchestra.run.smithi110:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-15T03:35:54.474 INFO:teuthology.orchestra.run.smithi110.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T03:35:54.474 INFO:teuthology.orchestra.run.smithi110.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T03:35:54.474 INFO:teuthology.orchestra.run.smithi110.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T03:35:54.474 INFO:teuthology.orchestra.run.smithi110.stdout: = reflink=1 2024-09-15T03:35:54.474 INFO:teuthology.orchestra.run.smithi110.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T03:35:54.475 INFO:teuthology.orchestra.run.smithi110.stdout: = sunit=0 swidth=0 blks 2024-09-15T03:35:54.475 INFO:teuthology.orchestra.run.smithi110.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T03:35:54.475 INFO:teuthology.orchestra.run.smithi110.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-15T03:35:54.475 INFO:teuthology.orchestra.run.smithi110.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T03:35:54.475 INFO:teuthology.orchestra.run.smithi110.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T03:35:54.477 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi110.front.sepia.ceph.com -o noatime 2024-09-15T03:35:54.477 DEBUG:teuthology.orchestra.run.smithi110:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-15T03:35:54.539 DEBUG:teuthology.orchestra.run.smithi110:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-15T03:35:54.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T03:35:54.547 INFO:teuthology.orchestra.run.smithi110.stderr:sudo: /sbin/restorecon: command not found 2024-09-15T03:35:54.547 DEBUG:teuthology.orchestra.run.smithi110:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-15T03:35:54.558 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-15T03:35:54.558 INFO:tasks.ceph:role: osd.3 2024-09-15T03:35:54.558 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi110.front.sepia.ceph.com 2024-09-15T03:35:54.558 DEBUG:teuthology.orchestra.run.smithi110:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-15T03:35:54.600 INFO:teuthology.orchestra.run.smithi110.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T03:35:54.600 INFO:teuthology.orchestra.run.smithi110.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T03:35:54.600 INFO:teuthology.orchestra.run.smithi110.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T03:35:54.600 INFO:teuthology.orchestra.run.smithi110.stdout: = reflink=1 2024-09-15T03:35:54.601 INFO:teuthology.orchestra.run.smithi110.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T03:35:54.601 INFO:teuthology.orchestra.run.smithi110.stdout: = sunit=0 swidth=0 blks 2024-09-15T03:35:54.601 INFO:teuthology.orchestra.run.smithi110.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T03:35:54.601 INFO:teuthology.orchestra.run.smithi110.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-15T03:35:54.601 INFO:teuthology.orchestra.run.smithi110.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T03:35:54.601 INFO:teuthology.orchestra.run.smithi110.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T03:35:54.602 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi110.front.sepia.ceph.com -o noatime 2024-09-15T03:35:54.603 DEBUG:teuthology.orchestra.run.smithi110:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-15T03:35:54.619 DEBUG:teuthology.orchestra.run.smithi110:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-15T03:35:54.673 INFO:teuthology.orchestra.run.smithi110.stderr:sudo: /sbin/restorecon: command not found 2024-09-15T03:35:54.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T03:35:54.674 DEBUG:teuthology.orchestra.run.smithi110:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T03:35:54.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:35:54.748+0000 7f5765967540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-15T03:35:54.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:35:54.748+0000 7f5765967540 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-15T03:35:54.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:35:54.748+0000 7f5765967540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-15T03:35:54.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:35:54.748+0000 7f5765967540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-15T03:35:54.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:35:54.748+0000 7f5765967540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-15T03:35:54.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:35:54.748+0000 7f5765967540 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-15T03:35:55.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:35:55.300+0000 7f5765967540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T03:35:56.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:35:56.080+0000 7f5765967540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T03:35:56.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:35:56.864+0000 7f5765967540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T03:35:57.737 DEBUG:teuthology.orchestra.run.smithi110:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-15T03:35:57.755 DEBUG:teuthology.orchestra.run.smithi110:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T03:35:57.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:35:57.788+0000 7fde01237540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-15T03:35:57.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:35:57.788+0000 7fde01237540 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-15T03:35:57.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:35:57.788+0000 7fde01237540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-15T03:35:57.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:35:57.788+0000 7fde01237540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-15T03:35:57.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:35:57.788+0000 7fde01237540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-15T03:35:57.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:35:57.788+0000 7fde01237540 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-15T03:35:58.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:35:58.348+0000 7fde01237540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T03:35:59.141 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:35:59.136+0000 7fde01237540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T03:35:59.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:35:59.928+0000 7fde01237540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T03:36:00.773 DEBUG:teuthology.orchestra.run.smithi110:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-15T03:36:00.790 DEBUG:teuthology.orchestra.run.smithi110:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T03:36:00.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:00.876+0000 7f10f3d08540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-15T03:36:00.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:00.876+0000 7f10f3d08540 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-15T03:36:00.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:00.876+0000 7f10f3d08540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-15T03:36:00.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:00.876+0000 7f10f3d08540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-15T03:36:00.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:00.876+0000 7f10f3d08540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-15T03:36:00.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:00.876+0000 7f10f3d08540 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-15T03:36:01.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:01.428+0000 7f10f3d08540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T03:36:02.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:02.212+0000 7f10f3d08540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T03:36:03.004 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:03.000+0000 7f10f3d08540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T03:36:03.844 DEBUG:teuthology.orchestra.run.smithi110:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-15T03:36:03.863 DEBUG:teuthology.orchestra.run.smithi110:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T03:36:03.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:03.896+0000 7fdba818c540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-15T03:36:03.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:03.896+0000 7fdba818c540 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-15T03:36:03.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:03.896+0000 7fdba818c540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-15T03:36:03.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:03.896+0000 7fdba818c540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-15T03:36:03.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:03.896+0000 7fdba818c540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-15T03:36:03.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:03.896+0000 7fdba818c540 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-15T03:36:04.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:04.444+0000 7fdba818c540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T03:36:05.229 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:05.224+0000 7fdba818c540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T03:36:06.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:06.020+0000 7fdba818c540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T03:36:06.861 DEBUG:teuthology.orchestra.run.smithi110:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-15T03:36:06.916 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-15T03:36:06.936 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-15T03:36:06.936 INFO:tasks.ceph:role: osd.4 2024-09-15T03:36:06.936 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi177.front.sepia.ceph.com 2024-09-15T03:36:06.937 DEBUG:teuthology.orchestra.run.smithi177:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-15T03:36:07.447 INFO:teuthology.orchestra.run.smithi177.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T03:36:07.448 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T03:36:07.448 INFO:teuthology.orchestra.run.smithi177.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T03:36:07.448 INFO:teuthology.orchestra.run.smithi177.stdout: = reflink=1 2024-09-15T03:36:07.448 INFO:teuthology.orchestra.run.smithi177.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T03:36:07.448 INFO:teuthology.orchestra.run.smithi177.stdout: = sunit=0 swidth=0 blks 2024-09-15T03:36:07.448 INFO:teuthology.orchestra.run.smithi177.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T03:36:07.448 INFO:teuthology.orchestra.run.smithi177.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-15T03:36:07.448 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T03:36:07.448 INFO:teuthology.orchestra.run.smithi177.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T03:36:07.450 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi177.front.sepia.ceph.com -o noatime 2024-09-15T03:36:07.450 DEBUG:teuthology.orchestra.run.smithi177:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-15T03:36:07.525 DEBUG:teuthology.orchestra.run.smithi177:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-15T03:36:07.536 INFO:teuthology.orchestra.run.smithi177.stderr:sudo: /sbin/restorecon: command not found 2024-09-15T03:36:07.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T03:36:07.537 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-15T03:36:07.553 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-15T03:36:07.553 INFO:tasks.ceph:role: osd.5 2024-09-15T03:36:07.553 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi177.front.sepia.ceph.com 2024-09-15T03:36:07.554 DEBUG:teuthology.orchestra.run.smithi177:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-15T03:36:08.079 INFO:teuthology.orchestra.run.smithi177.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T03:36:08.079 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T03:36:08.079 INFO:teuthology.orchestra.run.smithi177.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T03:36:08.079 INFO:teuthology.orchestra.run.smithi177.stdout: = reflink=1 2024-09-15T03:36:08.079 INFO:teuthology.orchestra.run.smithi177.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T03:36:08.079 INFO:teuthology.orchestra.run.smithi177.stdout: = sunit=0 swidth=0 blks 2024-09-15T03:36:08.079 INFO:teuthology.orchestra.run.smithi177.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T03:36:08.079 INFO:teuthology.orchestra.run.smithi177.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-15T03:36:08.080 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T03:36:08.080 INFO:teuthology.orchestra.run.smithi177.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T03:36:08.081 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi177.front.sepia.ceph.com -o noatime 2024-09-15T03:36:08.081 DEBUG:teuthology.orchestra.run.smithi177:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-15T03:36:08.136 DEBUG:teuthology.orchestra.run.smithi177:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-15T03:36:08.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T03:36:08.150 INFO:teuthology.orchestra.run.smithi177.stderr:sudo: /sbin/restorecon: command not found 2024-09-15T03:36:08.150 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-15T03:36:08.161 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-15T03:36:08.161 INFO:tasks.ceph:role: osd.6 2024-09-15T03:36:08.161 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi177.front.sepia.ceph.com 2024-09-15T03:36:08.164 DEBUG:teuthology.orchestra.run.smithi177:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-15T03:36:08.667 INFO:teuthology.orchestra.run.smithi177.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T03:36:08.667 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T03:36:08.667 INFO:teuthology.orchestra.run.smithi177.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T03:36:08.667 INFO:teuthology.orchestra.run.smithi177.stdout: = reflink=1 2024-09-15T03:36:08.667 INFO:teuthology.orchestra.run.smithi177.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T03:36:08.667 INFO:teuthology.orchestra.run.smithi177.stdout: = sunit=0 swidth=0 blks 2024-09-15T03:36:08.667 INFO:teuthology.orchestra.run.smithi177.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T03:36:08.667 INFO:teuthology.orchestra.run.smithi177.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-15T03:36:08.668 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T03:36:08.668 INFO:teuthology.orchestra.run.smithi177.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T03:36:08.669 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi177.front.sepia.ceph.com -o noatime 2024-09-15T03:36:08.670 DEBUG:teuthology.orchestra.run.smithi177:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-15T03:36:08.734 DEBUG:teuthology.orchestra.run.smithi177:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-15T03:36:08.747 INFO:teuthology.orchestra.run.smithi177.stderr:sudo: /sbin/restorecon: command not found 2024-09-15T03:36:08.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T03:36:08.749 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-15T03:36:08.807 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-15T03:36:08.807 INFO:tasks.ceph:role: osd.7 2024-09-15T03:36:08.807 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi177.front.sepia.ceph.com 2024-09-15T03:36:08.807 DEBUG:teuthology.orchestra.run.smithi177:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-15T03:36:09.305 INFO:teuthology.orchestra.run.smithi177.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T03:36:09.305 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T03:36:09.305 INFO:teuthology.orchestra.run.smithi177.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T03:36:09.305 INFO:teuthology.orchestra.run.smithi177.stdout: = reflink=1 2024-09-15T03:36:09.305 INFO:teuthology.orchestra.run.smithi177.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T03:36:09.305 INFO:teuthology.orchestra.run.smithi177.stdout: = sunit=0 swidth=0 blks 2024-09-15T03:36:09.305 INFO:teuthology.orchestra.run.smithi177.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T03:36:09.305 INFO:teuthology.orchestra.run.smithi177.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-15T03:36:09.306 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T03:36:09.306 INFO:teuthology.orchestra.run.smithi177.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T03:36:09.307 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi177.front.sepia.ceph.com -o noatime 2024-09-15T03:36:09.307 DEBUG:teuthology.orchestra.run.smithi177:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-15T03:36:09.366 DEBUG:teuthology.orchestra.run.smithi177:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-15T03:36:09.378 INFO:teuthology.orchestra.run.smithi177.stderr:sudo: /sbin/restorecon: command not found 2024-09-15T03:36:09.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T03:36:09.379 DEBUG:teuthology.orchestra.run.smithi177:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T03:36:09.445 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:09.439+0000 7f11ae425540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-15T03:36:09.446 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:09.439+0000 7f11ae425540 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-15T03:36:09.447 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:09.439+0000 7f11ae425540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-15T03:36:09.447 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:09.439+0000 7f11ae425540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-15T03:36:09.447 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:09.439+0000 7f11ae425540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-15T03:36:09.453 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:09.447+0000 7f11ae425540 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-15T03:36:10.021 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:10.015+0000 7f11ae425540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T03:36:10.799 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:10.795+0000 7f11ae425540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T03:36:11.578 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:11.571+0000 7f11ae425540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T03:36:12.435 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-15T03:36:12.452 DEBUG:teuthology.orchestra.run.smithi177:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T03:36:12.488 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:12.483+0000 7f6024d32540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-15T03:36:12.489 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:12.483+0000 7f6024d32540 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-15T03:36:12.489 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:12.483+0000 7f6024d32540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-15T03:36:12.489 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:12.483+0000 7f6024d32540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-15T03:36:12.489 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:12.483+0000 7f6024d32540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-15T03:36:12.489 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:12.483+0000 7f6024d32540 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-15T03:36:13.039 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:13.035+0000 7f6024d32540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T03:36:13.821 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:13.815+0000 7f6024d32540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T03:36:14.616 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:14.611+0000 7f6024d32540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T03:36:15.463 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-15T03:36:15.523 DEBUG:teuthology.orchestra.run.smithi177:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T03:36:15.560 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:15.555+0000 7fad8aaeb540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-15T03:36:15.561 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:15.555+0000 7fad8aaeb540 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-15T03:36:15.561 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:15.555+0000 7fad8aaeb540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-15T03:36:15.561 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:15.555+0000 7fad8aaeb540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-15T03:36:15.561 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:15.555+0000 7fad8aaeb540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-15T03:36:15.561 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:15.555+0000 7fad8aaeb540 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-15T03:36:16.111 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:16.107+0000 7fad8aaeb540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T03:36:16.901 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:16.895+0000 7fad8aaeb540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T03:36:17.682 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:17.675+0000 7fad8aaeb540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T03:36:18.542 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-15T03:36:18.559 DEBUG:teuthology.orchestra.run.smithi177:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T03:36:18.643 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:18.635+0000 7f9050d35540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-15T03:36:18.643 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:18.639+0000 7f9050d35540 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-15T03:36:18.643 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:18.639+0000 7f9050d35540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-15T03:36:18.643 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:18.639+0000 7f9050d35540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-15T03:36:18.643 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:18.639+0000 7f9050d35540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-15T03:36:18.644 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:18.639+0000 7f9050d35540 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-15T03:36:19.197 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:19.191+0000 7f9050d35540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T03:36:19.996 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:19.991+0000 7f9050d35540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T03:36:20.773 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:20.771+0000 7f9050d35540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T03:36:21.630 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-15T03:36:21.647 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-15T03:36:21.647 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-15T03:36:21.647 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-15T03:36:21.664 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-15T03:36:21.664 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-15T03:36:21.673 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-15T03:36:21.674 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-15T03:36:21.733 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-15T03:36:21.734 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-15T03:36:21.743 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-15T03:36:21.743 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-15T03:36:21.760 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-15T03:36:21.760 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-15T03:36:21.776 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-15T03:36:21.776 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-15T03:36:21.835 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-15T03:36:21.835 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-15T03:36:21.845 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-15T03:36:21.846 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-15T03:36:21.858 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-15T03:36:21.858 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-15T03:36:21.868 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-15T03:36:21.868 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-15T03:36:21.875 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-15T03:36:21.875 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-15T03:36:21.919 INFO:tasks.ceph:Adding keys to all mons... 2024-09-15T03:36:21.919 DEBUG:teuthology.orchestra.run.smithi110:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-15T03:36:21.922 DEBUG:teuthology.orchestra.run.smithi177:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-15T03:36:21.932 INFO:teuthology.orchestra.run.smithi110.stdout:[mgr.y] 2024-09-15T03:36:21.932 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQAZVuZmN79jORAADiqH9MiufsTPOPQXSzHeXg== 2024-09-15T03:36:21.933 INFO:teuthology.orchestra.run.smithi110.stdout:[osd.0] 2024-09-15T03:36:21.933 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQAaVuZm9IvKLBAAPY6Y9oc6JMymd0Z83mHltg== 2024-09-15T03:36:21.933 INFO:teuthology.orchestra.run.smithi110.stdout:[osd.1] 2024-09-15T03:36:21.933 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQAdVuZmSeUjLxAAMIYzMYwRAMMqy3IKL9GQ0Q== 2024-09-15T03:36:21.933 INFO:teuthology.orchestra.run.smithi110.stdout:[osd.2] 2024-09-15T03:36:21.933 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQAgVuZmeZRmNBAAFuHUDzqOz5fralPzkiegMQ== 2024-09-15T03:36:21.933 INFO:teuthology.orchestra.run.smithi110.stdout:[osd.3] 2024-09-15T03:36:21.933 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQAjVuZm1FqSNRAAag7EAY5czFtnoBkn1O2trw== 2024-09-15T03:36:21.933 INFO:teuthology.orchestra.run.smithi110.stdout:[mgr.x] 2024-09-15T03:36:21.933 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQAaVuZmM9ebABAAocxZGBTY6QcK3MlQbhBf+A== 2024-09-15T03:36:21.934 INFO:teuthology.orchestra.run.smithi110.stdout:[osd.4] 2024-09-15T03:36:21.934 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQApVuZmZfaSGhAAuDTIEKiqD6FWcfAT5WSUvg== 2024-09-15T03:36:21.934 INFO:teuthology.orchestra.run.smithi110.stdout:[osd.5] 2024-09-15T03:36:21.934 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQAsVuZmMZsfHRAA4wZ6wTEzX7dpx3+SU2pZyQ== 2024-09-15T03:36:21.934 INFO:teuthology.orchestra.run.smithi110.stdout:[osd.6] 2024-09-15T03:36:21.934 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQAvVuZmya1kIRAAJ1gvIvSaPChLzvAaoVRXbw== 2024-09-15T03:36:21.934 INFO:teuthology.orchestra.run.smithi110.stdout:[osd.7] 2024-09-15T03:36:21.934 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQAyVuZmGAhWJhAA2iCRKx464dJfe8vYX9pUmA== 2024-09-15T03:36:21.935 INFO:teuthology.orchestra.run.smithi110.stdout:[client.0] 2024-09-15T03:36:21.935 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQAaVuZmfSLsAhAAmvNa6wTYxjgRyKyRQCWsXA== 2024-09-15T03:36:21.935 INFO:teuthology.orchestra.run.smithi110.stdout:[client.1] 2024-09-15T03:36:21.935 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQAaVuZmEUr8BRAA2hIbabCwnqWxzWayWuyEAg== 2024-09-15T03:36:21.937 INFO:teuthology.orchestra.run.smithi177.stdout:[mgr.y] 2024-09-15T03:36:21.937 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQAZVuZmN79jORAADiqH9MiufsTPOPQXSzHeXg== 2024-09-15T03:36:21.937 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.0] 2024-09-15T03:36:21.937 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQAaVuZm9IvKLBAAPY6Y9oc6JMymd0Z83mHltg== 2024-09-15T03:36:21.937 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.1] 2024-09-15T03:36:21.937 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQAdVuZmSeUjLxAAMIYzMYwRAMMqy3IKL9GQ0Q== 2024-09-15T03:36:21.938 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.2] 2024-09-15T03:36:21.938 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQAgVuZmeZRmNBAAFuHUDzqOz5fralPzkiegMQ== 2024-09-15T03:36:21.938 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.3] 2024-09-15T03:36:21.938 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQAjVuZm1FqSNRAAag7EAY5czFtnoBkn1O2trw== 2024-09-15T03:36:21.938 INFO:teuthology.orchestra.run.smithi177.stdout:[mgr.x] 2024-09-15T03:36:21.938 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQAaVuZmM9ebABAAocxZGBTY6QcK3MlQbhBf+A== 2024-09-15T03:36:21.938 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.4] 2024-09-15T03:36:21.938 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQApVuZmZfaSGhAAuDTIEKiqD6FWcfAT5WSUvg== 2024-09-15T03:36:21.938 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.5] 2024-09-15T03:36:21.939 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQAsVuZmMZsfHRAA4wZ6wTEzX7dpx3+SU2pZyQ== 2024-09-15T03:36:21.939 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.6] 2024-09-15T03:36:21.939 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQAvVuZmya1kIRAAJ1gvIvSaPChLzvAaoVRXbw== 2024-09-15T03:36:21.939 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.7] 2024-09-15T03:36:21.939 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQAyVuZmGAhWJhAA2iCRKx464dJfe8vYX9pUmA== 2024-09-15T03:36:21.939 INFO:teuthology.orchestra.run.smithi177.stdout:[client.0] 2024-09-15T03:36:21.939 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQAaVuZmfSLsAhAAmvNa6wTYxjgRyKyRQCWsXA== 2024-09-15T03:36:21.939 INFO:teuthology.orchestra.run.smithi177.stdout:[client.1] 2024-09-15T03:36:21.939 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQAaVuZmEUr8BRAA2hIbabCwnqWxzWayWuyEAg== 2024-09-15T03:36:21.940 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-15T03:36:21.980 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-15T03:36:22.027 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T03:36:22.029 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T03:36:22.114 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T03:36:22.117 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T03:36:22.160 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T03:36:22.163 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T03:36:22.206 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T03:36:22.210 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T03:36:22.293 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-15T03:36:22.295 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-15T03:36:22.339 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T03:36:22.341 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T03:36:22.382 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T03:36:22.385 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T03:36:22.469 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T03:36:22.472 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T03:36:22.516 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T03:36:22.518 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T03:36:22.605 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-15T03:36:22.607 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-15T03:36:22.653 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-15T03:36:22.655 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-15T03:36:22.701 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-15T03:36:22.702 DEBUG:teuthology.orchestra.run.smithi110:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-15T03:36:22.719 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-15T03:36:22.770 DEBUG:teuthology.orchestra.run.smithi110:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-15T03:36:22.783 DEBUG:teuthology.orchestra.run.smithi110:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-15T03:36:22.841 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-15T03:36:22.894 DEBUG:teuthology.orchestra.run.smithi110:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-15T03:36:22.908 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-15T03:36:22.925 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-15T03:36:22.978 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-15T03:36:22.990 DEBUG:teuthology.orchestra.run.smithi110:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-15T03:36:22.992 DEBUG:teuthology.orchestra.run.smithi177:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-15T03:36:23.036 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-15T03:36:23.037 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-15T03:36:23.037 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-15T03:36:23.043 INFO:tasks.ceph.mon.a:Started 2024-09-15T03:36:23.043 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-15T03:36:23.043 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-15T03:36:23.045 INFO:tasks.ceph.mon.c:Started 2024-09-15T03:36:23.045 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-15T03:36:23.046 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-15T03:36:23.048 INFO:tasks.ceph.mon.b:Started 2024-09-15T03:36:23.048 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-15T03:36:23.048 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-15T03:36:23.048 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-15T03:36:23.050 INFO:tasks.ceph.mgr.y:Started 2024-09-15T03:36:23.051 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-15T03:36:23.051 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-15T03:36:23.053 INFO:tasks.ceph.mgr.x:Started 2024-09-15T03:36:23.053 INFO:tasks.ceph:Setting crush tunables to default 2024-09-15T03:36:23.053 DEBUG:teuthology.orchestra.run.smithi110:> sudo ceph --cluster ceph osd crush tunables default 2024-09-15T03:36:23.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:23.155+0000 7f6e81270700 1 Processor -- start 2024-09-15T03:36:23.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:23.155+0000 7f6e81270700 1 -- start start 2024-09-15T03:36:23.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:23.159+0000 7f6e81270700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f6e7c0ad070 0x7f6e7c0ab830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:23.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:23.159+0000 7f6e81270700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f6e7c13c300 0x7f6e7c13f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:23.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:23.159+0000 7f6e81270700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e7c13eb60 con 0x7f6e7c0ad440 2024-09-15T03:36:23.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:23.159+0000 7f6e81270700 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e7c0a4950 con 0x7f6e7c13c300 2024-09-15T03:36:23.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:23.159+0000 7f6e81270700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e7c13b5d0 con 0x7f6e7c0ad070 2024-09-15T03:36:23.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:23.159+0000 7f6e7b7fe700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f6e7c13c300 0x7f6e7c13f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:23.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:23.159+0000 7f6e7b7fe700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f6e7c13c300 0x7f6e7c13f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:40334/0 (socket says 172.21.15.110:40334) 2024-09-15T03:36:23.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:23.159+0000 7f6e7b7fe700 1 -- 172.21.15.110:0/4219727634 learned_addr learned my addr 172.21.15.110:0/4219727634 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:36:23.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:23.159+0000 7f6e7affd700 1 --2- 172.21.15.110:0/4219727634 >> v2:172.21.15.177:3300/0 conn(0x7f6e7c0ad070 0x7f6e7c0ab830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:23.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:23.159+0000 7f6e7b7fe700 1 -- 172.21.15.110:0/4219727634 >> v2:172.21.15.110:3300/0 conn(0x7f6e7c13c300 msgr2=0x7f6e7c13f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T03:36:23.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:23.159+0000 7f6e7b7fe700 1 -- 172.21.15.110:0/4219727634 >> v2:172.21.15.110:3300/0 conn(0x7f6e7c13c300 msgr2=0x7f6e7c13f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T03:36:23.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:23.159+0000 7f6e7b7fe700 1 --2- 172.21.15.110:0/4219727634 >> v2:172.21.15.110:3300/0 conn(0x7f6e7c13c300 0x7f6e7c13f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T03:36:23.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:23.159+0000 7f6e7b7fe700 1 --2- 172.21.15.110:0/4219727634 >> v2:172.21.15.110:3300/0 conn(0x7f6e7c13c300 0x7f6e7c13f060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T03:36:23.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:23.159+0000 7f6e7affd700 1 -- 172.21.15.110:0/4219727634 >> v2:172.21.15.177:3300/0 conn(0x7f6e7c0ad070 msgr2=0x7f6e7c0ab830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T03:36:23.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:23.159+0000 7f6e7affd700 1 -- 172.21.15.110:0/4219727634 >> v2:172.21.15.177:3300/0 conn(0x7f6e7c0ad070 msgr2=0x7f6e7c0ab830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T03:36:23.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:23.159+0000 7f6e7affd700 1 --2- 172.21.15.110:0/4219727634 >> v2:172.21.15.177:3300/0 conn(0x7f6e7c0ad070 0x7f6e7c0ab830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T03:36:23.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:23.159+0000 7f6e7affd700 1 --2- 172.21.15.110:0/4219727634 >> v2:172.21.15.177:3300/0 conn(0x7f6e7c0ad070 0x7f6e7c0ab830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T03:36:23.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:23.363+0000 7f6e7b7fe700 1 --2- 172.21.15.110:0/4219727634 >> v2:172.21.15.110:3300/0 conn(0x7f6e7c13c300 0x7f6e7c13f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:23.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:23.363+0000 7f6e7affd700 1 --2- 172.21.15.110:0/4219727634 >> v2:172.21.15.177:3300/0 conn(0x7f6e7c0ad070 0x7f6e7c0ab830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:23.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:23.363+0000 7f6e7b7fe700 1 -- 172.21.15.110:0/4219727634 >> v2:172.21.15.110:3300/0 conn(0x7f6e7c13c300 msgr2=0x7f6e7c13f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T03:36:23.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:23.363+0000 7f6e7b7fe700 1 -- 172.21.15.110:0/4219727634 >> v2:172.21.15.110:3300/0 conn(0x7f6e7c13c300 msgr2=0x7f6e7c13f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T03:36:23.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:23.363+0000 7f6e7b7fe700 1 --2- 172.21.15.110:0/4219727634 >> v2:172.21.15.110:3300/0 conn(0x7f6e7c13c300 0x7f6e7c13f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T03:36:23.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:23.363+0000 7f6e7b7fe700 1 --2- 172.21.15.110:0/4219727634 >> v2:172.21.15.110:3300/0 conn(0x7f6e7c13c300 0x7f6e7c13f060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T03:36:23.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:23.363+0000 7f6e7affd700 1 -- 172.21.15.110:0/4219727634 >> v2:172.21.15.177:3300/0 conn(0x7f6e7c0ad070 msgr2=0x7f6e7c0ab830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T03:36:23.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:23.363+0000 7f6e7affd700 1 -- 172.21.15.110:0/4219727634 >> v2:172.21.15.177:3300/0 conn(0x7f6e7c0ad070 msgr2=0x7f6e7c0ab830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T03:36:23.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:23.363+0000 7f6e7affd700 1 --2- 172.21.15.110:0/4219727634 >> v2:172.21.15.177:3300/0 conn(0x7f6e7c0ad070 0x7f6e7c0ab830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T03:36:23.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:23.363+0000 7f6e7affd700 1 --2- 172.21.15.110:0/4219727634 >> v2:172.21.15.177:3300/0 conn(0x7f6e7c0ad070 0x7f6e7c0ab830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T03:36:23.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:23.767+0000 7f6e7b7fe700 1 --2- 172.21.15.110:0/4219727634 >> v2:172.21.15.110:3300/0 conn(0x7f6e7c13c300 0x7f6e7c13f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:23.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:23.767+0000 7f6e7b7fe700 1 -- 172.21.15.110:0/4219727634 >> v2:172.21.15.110:3300/0 conn(0x7f6e7c13c300 msgr2=0x7f6e7c13f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T03:36:23.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:23.767+0000 7f6e7b7fe700 1 -- 172.21.15.110:0/4219727634 >> v2:172.21.15.110:3300/0 conn(0x7f6e7c13c300 msgr2=0x7f6e7c13f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T03:36:23.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:23.767+0000 7f6e7affd700 1 --2- 172.21.15.110:0/4219727634 >> v2:172.21.15.177:3300/0 conn(0x7f6e7c0ad070 0x7f6e7c0ab830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:23.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:23.767+0000 7f6e7b7fe700 1 --2- 172.21.15.110:0/4219727634 >> v2:172.21.15.110:3300/0 conn(0x7f6e7c13c300 0x7f6e7c13f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T03:36:23.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:23.767+0000 7f6e7b7fe700 1 --2- 172.21.15.110:0/4219727634 >> v2:172.21.15.110:3300/0 conn(0x7f6e7c13c300 0x7f6e7c13f060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T03:36:23.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:23.767+0000 7f6e7affd700 1 -- 172.21.15.110:0/4219727634 >> v2:172.21.15.177:3300/0 conn(0x7f6e7c0ad070 msgr2=0x7f6e7c0ab830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T03:36:23.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:23.767+0000 7f6e7affd700 1 -- 172.21.15.110:0/4219727634 >> v2:172.21.15.177:3300/0 conn(0x7f6e7c0ad070 msgr2=0x7f6e7c0ab830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T03:36:23.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:23.767+0000 7f6e7affd700 1 --2- 172.21.15.110:0/4219727634 >> v2:172.21.15.177:3300/0 conn(0x7f6e7c0ad070 0x7f6e7c0ab830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T03:36:23.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:23.767+0000 7f6e7affd700 1 --2- 172.21.15.110:0/4219727634 >> v2:172.21.15.177:3300/0 conn(0x7f6e7c0ad070 0x7f6e7c0ab830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T03:36:24.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:24.571+0000 7f6e7b7fe700 1 --2- 172.21.15.110:0/4219727634 >> v2:172.21.15.110:3300/0 conn(0x7f6e7c13c300 0x7f6e7c13f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:24.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:24.571+0000 7f6e7b7fe700 1 -- 172.21.15.110:0/4219727634 >> v2:172.21.15.110:3300/0 conn(0x7f6e7c13c300 msgr2=0x7f6e7c13f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T03:36:24.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:24.571+0000 7f6e7b7fe700 1 -- 172.21.15.110:0/4219727634 >> v2:172.21.15.110:3300/0 conn(0x7f6e7c13c300 msgr2=0x7f6e7c13f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T03:36:24.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:24.571+0000 7f6e7b7fe700 1 --2- 172.21.15.110:0/4219727634 >> v2:172.21.15.110:3300/0 conn(0x7f6e7c13c300 0x7f6e7c13f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T03:36:24.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:24.571+0000 7f6e7affd700 1 --2- 172.21.15.110:0/4219727634 >> v2:172.21.15.177:3300/0 conn(0x7f6e7c0ad070 0x7f6e7c0ab830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:24.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:24.571+0000 7f6e7b7fe700 1 --2- 172.21.15.110:0/4219727634 >> v2:172.21.15.110:3300/0 conn(0x7f6e7c13c300 0x7f6e7c13f060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T03:36:24.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:24.571+0000 7f6e7affd700 1 -- 172.21.15.110:0/4219727634 >> v2:172.21.15.177:3300/0 conn(0x7f6e7c0ad070 msgr2=0x7f6e7c0ab830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T03:36:24.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:24.571+0000 7f6e7affd700 1 -- 172.21.15.110:0/4219727634 >> v2:172.21.15.177:3300/0 conn(0x7f6e7c0ad070 msgr2=0x7f6e7c0ab830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T03:36:24.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:24.571+0000 7f6e7affd700 1 --2- 172.21.15.110:0/4219727634 >> v2:172.21.15.177:3300/0 conn(0x7f6e7c0ad070 0x7f6e7c0ab830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T03:36:24.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:24.571+0000 7f6e7affd700 1 --2- 172.21.15.110:0/4219727634 >> v2:172.21.15.177:3300/0 conn(0x7f6e7c0ad070 0x7f6e7c0ab830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T03:36:26.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:26.159+0000 7f6e78ff9700 1 -- 172.21.15.110:0/4219727634 >> v2:172.21.15.177:3300/0 conn(0x7f6e7c0ad070 msgr2=0x7f6e7c0ab830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:36:26.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:26.159+0000 7f6e78ff9700 1 --2- 172.21.15.110:0/4219727634 >> v2:172.21.15.177:3300/0 conn(0x7f6e7c0ad070 0x7f6e7c0ab830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:26.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:26.159+0000 7f6e78ff9700 1 -- 172.21.15.110:0/4219727634 >> v2:172.21.15.110:3300/0 conn(0x7f6e7c13c300 msgr2=0x7f6e7c13f060 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:36:26.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:26.159+0000 7f6e78ff9700 1 --2- 172.21.15.110:0/4219727634 >> v2:172.21.15.110:3300/0 conn(0x7f6e7c13c300 0x7f6e7c13f060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:26.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:26.159+0000 7f6e78ff9700 1 -- 172.21.15.110:0/4219727634 >> v1:172.21.15.177:6789/0 conn(0x7f6e7c0ad440 legacy=0x7f6e7c13bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:26.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:26.159+0000 7f6e78ff9700 1 --2- 172.21.15.110:0/4219727634 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f6e680072e0 0x7f6e68009790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:26.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:26.159+0000 7f6e78ff9700 1 -- 172.21.15.110:0/4219727634 --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e7c0aaf90 con 0x7f6e680036e0 2024-09-15T03:36:26.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:26.159+0000 7f6e78ff9700 1 -- 172.21.15.110:0/4219727634 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c0a4950 con 0x7f6e680072e0 2024-09-15T03:36:26.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:26.159+0000 7f6e7a7fc700 1 --2- 172.21.15.110:0/4219727634 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f6e680072e0 0x7f6e68009790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:26.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:26.159+0000 7f6e7a7fc700 1 -- 172.21.15.110:0/4219727634 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f6e680072e0 msgr2=0x7f6e68009790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T03:36:26.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:26.159+0000 7f6e7a7fc700 1 -- 172.21.15.110:0/4219727634 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f6e680072e0 msgr2=0x7f6e68009790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T03:36:26.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:26.159+0000 7f6e7a7fc700 1 --2- 172.21.15.110:0/4219727634 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f6e680072e0 0x7f6e68009790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T03:36:26.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:26.159+0000 7f6e7a7fc700 1 --2- 172.21.15.110:0/4219727634 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f6e680072e0 0x7f6e68009790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T03:36:26.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:26.363+0000 7f6e7a7fc700 1 --2- 172.21.15.110:0/4219727634 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f6e680072e0 0x7f6e68009790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:26.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:26.363+0000 7f6e7a7fc700 1 -- 172.21.15.110:0/4219727634 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f6e680072e0 msgr2=0x7f6e68009790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T03:36:26.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:26.363+0000 7f6e7a7fc700 1 -- 172.21.15.110:0/4219727634 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f6e680072e0 msgr2=0x7f6e68009790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T03:36:26.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:26.363+0000 7f6e7a7fc700 1 --2- 172.21.15.110:0/4219727634 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f6e680072e0 0x7f6e68009790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T03:36:26.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:26.363+0000 7f6e7a7fc700 1 --2- 172.21.15.110:0/4219727634 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f6e680072e0 0x7f6e68009790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T03:36:26.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:26.767+0000 7f6e7a7fc700 1 --2- 172.21.15.110:0/4219727634 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f6e680072e0 0x7f6e68009790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:26.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:26.767+0000 7f6e7a7fc700 1 -- 172.21.15.110:0/4219727634 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f6e680072e0 msgr2=0x7f6e68009790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T03:36:26.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:26.767+0000 7f6e7a7fc700 1 -- 172.21.15.110:0/4219727634 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f6e680072e0 msgr2=0x7f6e68009790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T03:36:26.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:26.767+0000 7f6e7a7fc700 1 --2- 172.21.15.110:0/4219727634 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f6e680072e0 0x7f6e68009790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T03:36:26.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:26.767+0000 7f6e7a7fc700 1 --2- 172.21.15.110:0/4219727634 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f6e680072e0 0x7f6e68009790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T03:36:27.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:27.571+0000 7f6e7a7fc700 1 --2- 172.21.15.110:0/4219727634 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f6e680072e0 0x7f6e68009790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:27.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:27.571+0000 7f6e7a7fc700 1 -- 172.21.15.110:0/4219727634 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f6e680072e0 msgr2=0x7f6e68009790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T03:36:27.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:27.571+0000 7f6e7a7fc700 1 -- 172.21.15.110:0/4219727634 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f6e680072e0 msgr2=0x7f6e68009790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T03:36:27.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:27.571+0000 7f6e7a7fc700 1 --2- 172.21.15.110:0/4219727634 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f6e680072e0 0x7f6e68009790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T03:36:27.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:27.571+0000 7f6e7a7fc700 1 --2- 172.21.15.110:0/4219727634 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f6e680072e0 0x7f6e68009790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T03:36:28.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.391+0000 7f6e79ffb700 1 -- 172.21.15.110:0/4219727634 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3657665121 0 0) 0x7f6e7c0aaf90 con 0x7f6e680036e0 2024-09-15T03:36:28.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.391+0000 7f6e79ffb700 1 -- 172.21.15.110:0/4219727634 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e64003570 con 0x7f6e680036e0 2024-09-15T03:36:28.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.395+0000 7f6e79ffb700 1 -- 172.21.15.110:0/4219727634 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3371388091 0 0) 0x7f6e64003570 con 0x7f6e680036e0 2024-09-15T03:36:28.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.395+0000 7f6e79ffb700 1 -- 172.21.15.110:0/4219727634 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f6e680072e0 msgr2=0x7f6e68009790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:36:28.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.395+0000 7f6e79ffb700 1 --2- 172.21.15.110:0/4219727634 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f6e680072e0 0x7f6e68009790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:28.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.395+0000 7f6e79ffb700 1 -- 172.21.15.110:0/4219727634 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e6800af60 con 0x7f6e680036e0 2024-09-15T03:36:28.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.395+0000 7f6e79ffb700 1 -- 172.21.15.110:0/4219727634 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 644009436 0 0) 0x7f6e6c00d300 con 0x7f6e680036e0 2024-09-15T03:36:28.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.395+0000 7f6e79ffb700 1 -- 172.21.15.110:0/4219727634 >> v1:172.21.15.110:6789/0 conn(0x7f6e680036e0 legacy=0x7f6e68005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:28.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.395+0000 7f6e79ffb700 1 --2- 172.21.15.110:0/4219727634 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f6e680072e0 0x7f6e64003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:28.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.395+0000 7f6e79ffb700 1 --2- 172.21.15.110:0/4219727634 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e64004070 0x7f6e64004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:28.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.395+0000 7f6e79ffb700 1 --2- 172.21.15.110:0/4219727634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e64004d50 0x7f6e640051a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:28.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.395+0000 7f6e79ffb700 1 -- 172.21.15.110:0/4219727634 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c0a4950 con 0x7f6e64004d50 2024-09-15T03:36:28.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.395+0000 7f6e7b7fe700 1 -- 172.21.15.110:0/4219727634 reap_dead start 2024-09-15T03:36:28.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.395+0000 7f6e7affd700 1 --2- 172.21.15.110:0/4219727634 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f6e680072e0 0x7f6e64003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:28.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.395+0000 7f6e79ffb700 1 -- 172.21.15.110:0/4219727634 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6400bcc0 con 0x7f6e64004070 2024-09-15T03:36:28.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.395+0000 7f6e79ffb700 1 -- 172.21.15.110:0/4219727634 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6400be80 con 0x7f6e680072e0 2024-09-15T03:36:28.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.395+0000 7f6e7b7fe700 1 --2- 172.21.15.110:0/4219727634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e64004d50 0x7f6e640051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:28.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.395+0000 7f6e7a7fc700 1 --2- 172.21.15.110:0/4219727634 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e64004070 0x7f6e64004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:28.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.395+0000 7f6e7affd700 1 -- 172.21.15.110:0/4219727634 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e64004070 msgr2=0x7f6e64004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:36:28.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.395+0000 7f6e7affd700 1 --2- 172.21.15.110:0/4219727634 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e64004070 0x7f6e64004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:28.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.395+0000 7f6e7affd700 1 -- 172.21.15.110:0/4219727634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e64004d50 msgr2=0x7f6e640051a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:36:28.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.395+0000 7f6e7affd700 1 --2- 172.21.15.110:0/4219727634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e64004d50 0x7f6e640051a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:28.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.395+0000 7f6e7affd700 1 -- 172.21.15.110:0/4219727634 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e64003570 con 0x7f6e680072e0 2024-09-15T03:36:28.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.395+0000 7f6e7a7fc700 1 --2- 172.21.15.110:0/4219727634 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e64004070 0x7f6e64004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T03:36:28.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.395+0000 7f6e7affd700 1 --2- 172.21.15.110:0/4219727634 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f6e680072e0 0x7f6e64003a60 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c0133e0 tx=0x7f6e6c008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20e44ee5565ea23b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:28.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.395+0000 7f6e79ffb700 1 -- 172.21.15.110:0/4219727634 <== mon.2 v2:172.21.15.110:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e6c013c70 con 0x7f6e680072e0 2024-09-15T03:36:28.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.395+0000 7f6e79ffb700 1 -- 172.21.15.110:0/4219727634 <== mon.2 v2:172.21.15.110:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e6c013df0 con 0x7f6e680072e0 2024-09-15T03:36:28.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.395+0000 7f6e79ffb700 1 -- 172.21.15.110:0/4219727634 <== mon.2 v2:172.21.15.110:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e6c0166b0 con 0x7f6e680072e0 2024-09-15T03:36:28.404 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.399+0000 7f6e81270700 1 -- 172.21.15.110:0/4219727634 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f6e680072e0 msgr2=0x7f6e64003a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:28.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.399+0000 7f6e81270700 1 --2- 172.21.15.110:0/4219727634 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f6e680072e0 0x7f6e64003a60 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c0133e0 tx=0x7f6e6c008040 comp rx=0 tx=0).stop 2024-09-15T03:36:28.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.399+0000 7f6e81270700 1 -- 172.21.15.110:0/4219727634 shutdown_connections 2024-09-15T03:36:28.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.399+0000 7f6e81270700 1 --2- 172.21.15.110:0/4219727634 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e64004070 0x7f6e64004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:28.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.399+0000 7f6e81270700 1 --2- 172.21.15.110:0/4219727634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e64004d50 0x7f6e640051a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:28.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.399+0000 7f6e81270700 1 --2- 172.21.15.110:0/4219727634 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f6e680072e0 0x7f6e64003a60 secure :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c0133e0 tx=0x7f6e6c008040 comp rx=0 tx=0).stop 2024-09-15T03:36:28.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.399+0000 7f6e81270700 1 -- 172.21.15.110:0/4219727634 >> 172.21.15.110:0/4219727634 conn(0x7f6e7c0b00d0 msgr2=0x7f6e7c0b04d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:28.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.399+0000 7f6e81270700 1 -- 172.21.15.110:0/4219727634 shutdown_connections 2024-09-15T03:36:28.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.399+0000 7f6e81270700 1 -- 172.21.15.110:0/4219727634 wait complete. 2024-09-15T03:36:28.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.399+0000 7f6e81270700 1 Processor -- start 2024-09-15T03:36:28.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.399+0000 7f6e81270700 1 -- start start 2024-09-15T03:36:28.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.403+0000 7f6e81270700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e64004070 0x7f6e7c1509b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:28.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.403+0000 7f6e81270700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f6e64004d50 0x7f6e7c150ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:28.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.403+0000 7f6e81270700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e7c13fce0 0x7f6e7c136060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:28.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.403+0000 7f6e81270700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c06e650 con 0x7f6e7c13fce0 2024-09-15T03:36:28.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.403+0000 7f6e81270700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c06dac0 con 0x7f6e64004070 2024-09-15T03:36:28.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.403+0000 7f6e81270700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c06e8a0 con 0x7f6e64004d50 2024-09-15T03:36:28.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.403+0000 7f6e7b7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e7c13fce0 0x7f6e7c136060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:28.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.403+0000 7f6e7b7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e7c13fce0 0x7f6e7c136060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52908/0 (socket says 172.21.15.110:52908) 2024-09-15T03:36:28.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.403+0000 7f6e7b7fe700 1 -- 172.21.15.110:0/278690583 learned_addr learned my addr 172.21.15.110:0/278690583 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:36:28.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.403+0000 7f6e7affd700 1 --2- 172.21.15.110:0/278690583 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e64004070 0x7f6e7c1509b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:28.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.403+0000 7f6e7a7fc700 1 --2- 172.21.15.110:0/278690583 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f6e64004d50 0x7f6e7c150ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:28.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.403+0000 7f6e7b7fe700 1 -- 172.21.15.110:0/278690583 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f6e64004d50 msgr2=0x7f6e7c150ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:28.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.403+0000 7f6e7b7fe700 1 --2- 172.21.15.110:0/278690583 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f6e64004d50 0x7f6e7c150ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:28.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.403+0000 7f6e7b7fe700 1 -- 172.21.15.110:0/278690583 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e64004070 msgr2=0x7f6e7c1509b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:28.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.403+0000 7f6e7b7fe700 1 --2- 172.21.15.110:0/278690583 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e64004070 0x7f6e7c1509b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:28.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.403+0000 7f6e7b7fe700 1 -- 172.21.15.110:0/278690583 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e7c151530 con 0x7f6e7c13fce0 2024-09-15T03:36:28.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.403+0000 7f6e7a7fc700 1 --2- 172.21.15.110:0/278690583 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f6e64004d50 0x7f6e7c150ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T03:36:28.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.403+0000 7f6e7b7fe700 1 --2- 172.21.15.110:0/278690583 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e7c13fce0 0x7f6e7c136060 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6e74009650 tx=0x7f6e7400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:28.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.403+0000 7f6e62ffd700 1 -- 172.21.15.110:0/278690583 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e74013980 con 0x7f6e7c13fce0 2024-09-15T03:36:28.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.403+0000 7f6e62ffd700 1 -- 172.21.15.110:0/278690583 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e7400f040 con 0x7f6e7c13fce0 2024-09-15T03:36:28.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.403+0000 7f6e62ffd700 1 -- 172.21.15.110:0/278690583 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e74013d10 con 0x7f6e7c13fce0 2024-09-15T03:36:28.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.403+0000 7f6e81270700 1 -- 172.21.15.110:0/278690583 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e7c1407a0 con 0x7f6e7c13fce0 2024-09-15T03:36:28.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.403+0000 7f6e81270700 1 -- 172.21.15.110:0/278690583 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e7c136740 con 0x7f6e7c13fce0 2024-09-15T03:36:28.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.403+0000 7f6e62ffd700 1 -- 172.21.15.110:0/278690583 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f6e74018660 con 0x7f6e7c13fce0 2024-09-15T03:36:28.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.403+0000 7f6e62ffd700 1 -- 172.21.15.110:0/278690583 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f6e740217a0 con 0x7f6e7c13fce0 2024-09-15T03:36:28.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.407+0000 7f6e81270700 1 -- 172.21.15.110:0/278690583 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e7c064c70 con 0x7f6e7c13fce0 2024-09-15T03:36:28.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.411+0000 7f6e62ffd700 1 -- 172.21.15.110:0/278690583 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f6e74026060 con 0x7f6e7c13fce0 2024-09-15T03:36:28.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.479+0000 7f6e81270700 1 -- 172.21.15.110:0/278690583 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f6e7c0ab520 con 0x7f6e7c13fce0 2024-09-15T03:36:28.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.483+0000 7f6e62ffd700 1 -- 172.21.15.110:0/278690583 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f6e7402f090 con 0x7f6e7c13fce0 2024-09-15T03:36:28.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.483+0000 7f6e81270700 1 -- 172.21.15.110:0/278690583 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e7c13fce0 msgr2=0x7f6e7c136060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:28.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.483+0000 7f6e81270700 1 --2- 172.21.15.110:0/278690583 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e7c13fce0 0x7f6e7c136060 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6e74009650 tx=0x7f6e7400b040 comp rx=0 tx=0).stop 2024-09-15T03:36:28.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.483+0000 7f6e81270700 1 -- 172.21.15.110:0/278690583 shutdown_connections 2024-09-15T03:36:28.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.483+0000 7f6e81270700 1 --2- 172.21.15.110:0/278690583 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e7c13fce0 0x7f6e7c136060 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:28.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.483+0000 7f6e81270700 1 --2- 172.21.15.110:0/278690583 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f6e64004d50 0x7f6e7c150ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:28.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.483+0000 7f6e81270700 1 --2- 172.21.15.110:0/278690583 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e64004070 0x7f6e7c1509b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:28.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.483+0000 7f6e81270700 1 -- 172.21.15.110:0/278690583 >> 172.21.15.110:0/278690583 conn(0x7f6e7c0b00d0 msgr2=0x7f6e7c0a9260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:28.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.487+0000 7f6e81270700 1 -- 172.21.15.110:0/278690583 shutdown_connections 2024-09-15T03:36:28.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:28.487+0000 7f6e81270700 1 -- 172.21.15.110:0/278690583 wait complete. 2024-09-15T03:36:28.493 INFO:teuthology.orchestra.run.smithi110.stderr:adjusted tunables profile to default 2024-09-15T03:36:28.502 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-15T03:36:28.502 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-15T03:36:28.502 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-15T03:36:28.512 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-15T03:36:28.512 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-15T03:36:28.522 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-15T03:36:28.522 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-15T03:36:28.538 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-15T03:36:28.538 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-15T03:36:28.547 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-15T03:36:28.547 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-15T03:36:28.562 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-15T03:36:28.563 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-15T03:36:28.590 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-15T03:36:28.587+0000 7fa4cdfbb000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-15T03:36:28.622 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-15T03:36:28.622 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-15T03:36:28.631 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-15T03:36:28.631 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-15T03:36:28.641 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd new 09d93c1d-8ec1-42e8-8b59-5e20958830d6 0 2024-09-15T03:36:28.750 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.747+0000 7f0cc3d9b700 1 Processor -- start 2024-09-15T03:36:28.750 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.747+0000 7f0cc3d9b700 1 -- start start 2024-09-15T03:36:28.751 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.751+0000 7f0cc3d9b700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f0cbc0b1450 0x7f0cbc140cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:28.752 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.751+0000 7f0cc3d9b700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f0cbc141210 0x7f0cbc0aeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:28.752 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.751+0000 7f0cc3d9b700 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cbc0a5370 con 0x7f0cbc0af030 2024-09-15T03:36:28.752 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.751+0000 7f0cc3d9b700 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cbc0b1820 con 0x7f0cbc0b1450 2024-09-15T03:36:28.752 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.751+0000 7f0cc3d9b700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cbc141660 con 0x7f0cbc141210 2024-09-15T03:36:28.752 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.751+0000 7f0cc1336700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f0cbc141210 0x7f0cbc0aeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:28.752 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.751+0000 7f0cc1b37700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f0cbc0b1450 0x7f0cbc140cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:28.752 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.751+0000 7f0cc1336700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f0cbc141210 0x7f0cbc0aeaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3301/0 says I am v2:172.21.15.177:59098/0 (socket says 172.21.15.177:59098) 2024-09-15T03:36:28.752 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.751+0000 7f0cc1336700 1 -- 172.21.15.177:0/3936721989 learned_addr learned my addr 172.21.15.177:0/3936721989 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-15T03:36:28.754 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.751+0000 7f0cc1336700 1 -- 172.21.15.177:0/3936721989 >> v2:172.21.15.110:3300/0 conn(0x7f0cbc0b1450 msgr2=0x7f0cbc140cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:36:28.754 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.751+0000 7f0cc1336700 1 --2- 172.21.15.177:0/3936721989 >> v2:172.21.15.110:3300/0 conn(0x7f0cbc0b1450 0x7f0cbc140cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:28.754 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.751+0000 7f0cc1336700 1 -- 172.21.15.177:0/3936721989 >> v1:172.21.15.110:6789/0 conn(0x7f0cbc0af030 legacy=0x7f0cbc13a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:28.754 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.751+0000 7f0cc1336700 1 -- 172.21.15.177:0/3936721989 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cbc0a6760 con 0x7f0cbc141210 2024-09-15T03:36:28.754 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.751+0000 7f0cc1336700 1 --2- 172.21.15.177:0/3936721989 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f0cbc141210 0x7f0cbc0aeaf0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0cac0099b0 tx=0x7f0cac009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8afb1fd72de49340 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:28.755 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.751+0000 7f0cc1336700 1 -- 172.21.15.177:0/3936721989 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f0cbc141210 msgr2=0x7f0cbc0aeaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-09-15T03:36:28.755 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.751+0000 7f0cc1336700 1 -- 172.21.15.177:0/3936721989 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f0cbc141210 msgr2=0x7f0cbc0aeaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T03:36:28.755 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.751+0000 7f0cc1336700 1 --2- 172.21.15.177:0/3936721989 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f0cbc141210 0x7f0cbc0aeaf0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0cac0099b0 tx=0x7f0cac009250 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T03:36:28.755 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.751+0000 7f0cc1336700 1 --2- 172.21.15.177:0/3936721989 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f0cbc141210 0x7f0cbc0aeaf0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0cac0099b0 tx=0x7f0cac009250 comp rx=0 tx=0).stop 2024-09-15T03:36:28.755 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.751+0000 7f0cc0b35700 1 -- 172.21.15.177:0/3936721989 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f0cbc141210 msgr2=0x7f0cbc0aeaf0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:36:28.755 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.751+0000 7f0cc0b35700 1 --2- 172.21.15.177:0/3936721989 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f0cbc141210 0x7f0cbc0aeaf0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:28.755 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.751+0000 7f0cc0b35700 1 --2- 172.21.15.177:0/3936721989 >> v2:172.21.15.177:3300/0 conn(0x7f0ca80074d0 0x7f0ca8009980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:28.755 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.751+0000 7f0cc0b35700 1 -- 172.21.15.177:0/3936721989 --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ca800a250 con 0x7f0ca8003550 2024-09-15T03:36:28.755 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.751+0000 7f0cc0b35700 1 -- 172.21.15.177:0/3936721989 --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ca800b410 con 0x7f0ca80074d0 2024-09-15T03:36:28.755 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.751+0000 7f0cc1336700 1 --2- 172.21.15.177:0/3936721989 >> v2:172.21.15.177:3300/0 conn(0x7f0ca80074d0 0x7f0ca8009980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:28.755 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.751+0000 7f0cc1336700 1 -- 172.21.15.177:0/3936721989 >> v1:172.21.15.177:6789/0 conn(0x7f0ca8003550 legacy=0x7f0ca80059c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:28.755 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.751+0000 7f0cc1336700 1 -- 172.21.15.177:0/3936721989 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ca800b640 con 0x7f0ca80074d0 2024-09-15T03:36:28.755 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.751+0000 7f0cc1336700 1 --2- 172.21.15.177:0/3936721989 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ca80074d0 0x7f0ca8009980 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0cac016000 tx=0x7f0cac017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30de53fce53f738f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:28.756 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.751+0000 7f0cc0b35700 1 -- 172.21.15.177:0/3936721989 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cac020070 con 0x7f0ca80074d0 2024-09-15T03:36:28.756 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.751+0000 7f0cc0b35700 1 -- 172.21.15.177:0/3936721989 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cac00bd40 con 0x7f0ca80074d0 2024-09-15T03:36:28.756 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.751+0000 7f0cc0b35700 1 -- 172.21.15.177:0/3936721989 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cac01a650 con 0x7f0ca80074d0 2024-09-15T03:36:28.759 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.755+0000 7f0cc3d9b700 1 -- 172.21.15.177:0/3936721989 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ca80074d0 msgr2=0x7f0ca8009980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:28.759 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.755+0000 7f0cc3d9b700 1 --2- 172.21.15.177:0/3936721989 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ca80074d0 0x7f0ca8009980 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0cac016000 tx=0x7f0cac017000 comp rx=0 tx=0).stop 2024-09-15T03:36:28.759 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.755+0000 7f0cc2338700 1 -- 172.21.15.177:0/3936721989 reap_dead start 2024-09-15T03:36:28.759 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.755+0000 7f0cc3d9b700 1 -- 172.21.15.177:0/3936721989 shutdown_connections 2024-09-15T03:36:28.759 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.755+0000 7f0cc3d9b700 1 --2- 172.21.15.177:0/3936721989 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ca80074d0 0x7f0ca8009980 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:28.760 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.755+0000 7f0cc3d9b700 1 -- 172.21.15.177:0/3936721989 >> 172.21.15.177:0/3936721989 conn(0x7f0cbc06f140 msgr2=0x7f0cbc06f540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:28.760 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.755+0000 7f0cc3d9b700 1 -- 172.21.15.177:0/3936721989 shutdown_connections 2024-09-15T03:36:28.760 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.755+0000 7f0cc3d9b700 1 -- 172.21.15.177:0/3936721989 wait complete. 2024-09-15T03:36:28.760 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.755+0000 7f0cc3d9b700 1 Processor -- start 2024-09-15T03:36:28.760 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.755+0000 7f0cc3d9b700 1 -- start start 2024-09-15T03:36:28.760 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.755+0000 7f0cc3d9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0ca80074d0 0x7f0cbc0a8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:28.760 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.755+0000 7f0cc3d9b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0cbc14d850 0x7f0cbc14dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:28.760 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.755+0000 7f0cc3d9b700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f0cbc14e200 0x7f0cbc150a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:28.760 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.755+0000 7f0cc3d9b700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cbc0a8640 con 0x7f0ca80074d0 2024-09-15T03:36:28.761 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.755+0000 7f0cc3d9b700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cbc0a96f0 con 0x7f0cbc14d850 2024-09-15T03:36:28.761 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.755+0000 7f0cc3d9b700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cbc0a99d0 con 0x7f0cbc14e200 2024-09-15T03:36:28.761 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.755+0000 7f0cc1336700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0cbc14d850 0x7f0cbc14dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:28.761 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.755+0000 7f0cc1336700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0cbc14d850 0x7f0cbc14dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:54048/0 (socket says 172.21.15.177:54048) 2024-09-15T03:36:28.761 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.755+0000 7f0cc1336700 1 -- 172.21.15.177:0/1864436214 learned_addr learned my addr 172.21.15.177:0/1864436214 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-15T03:36:28.761 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.755+0000 7f0cc2338700 1 --2- 172.21.15.177:0/1864436214 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f0cbc14e200 0x7f0cbc150a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:28.761 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.755+0000 7f0cc1b37700 1 --2- 172.21.15.177:0/1864436214 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0ca80074d0 0x7f0cbc0a8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:28.762 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.755+0000 7f0cc1336700 1 -- 172.21.15.177:0/1864436214 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f0cbc14e200 msgr2=0x7f0cbc150a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:28.762 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.755+0000 7f0cc1336700 1 --2- 172.21.15.177:0/1864436214 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f0cbc14e200 0x7f0cbc150a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:28.762 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.755+0000 7f0cc1336700 1 -- 172.21.15.177:0/1864436214 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0ca80074d0 msgr2=0x7f0cbc0a8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:28.762 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.755+0000 7f0cc1336700 1 --2- 172.21.15.177:0/1864436214 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0ca80074d0 0x7f0cbc0a8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:28.762 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.755+0000 7f0cc1336700 1 -- 172.21.15.177:0/1864436214 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cbc13def0 con 0x7f0cbc14d850 2024-09-15T03:36:28.762 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.755+0000 7f0cc1336700 1 --2- 172.21.15.177:0/1864436214 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0cbc14d850 0x7f0cbc14dcc0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0cac00b970 tx=0x7f0cac01d070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:28.762 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.755+0000 7f0cb1ffb700 1 -- 172.21.15.177:0/1864436214 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cac020040 con 0x7f0cbc14d850 2024-09-15T03:36:28.762 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.755+0000 7f0cb1ffb700 1 -- 172.21.15.177:0/1864436214 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cac006bc0 con 0x7f0cbc14d850 2024-09-15T03:36:28.762 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.755+0000 7f0cc3d9b700 1 -- 172.21.15.177:0/1864436214 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cbc0a7300 con 0x7f0cbc14d850 2024-09-15T03:36:28.762 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.755+0000 7f0cb1ffb700 1 -- 172.21.15.177:0/1864436214 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cac02d880 con 0x7f0cbc14d850 2024-09-15T03:36:28.762 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.755+0000 7f0cc3d9b700 1 -- 172.21.15.177:0/1864436214 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cbc151150 con 0x7f0cbc14d850 2024-09-15T03:36:28.762 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.759+0000 7f0cb1ffb700 1 -- 172.21.15.177:0/1864436214 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f0cac02b070 con 0x7f0cbc14d850 2024-09-15T03:36:28.762 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.759+0000 7f0cb1ffb700 1 -- 172.21.15.177:0/1864436214 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 721+0+0 (secure 0 0 0) 0x7f0cac011530 con 0x7f0cbc14d850 2024-09-15T03:36:28.765 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.759+0000 7f0cc3d9b700 1 -- 172.21.15.177:0/1864436214 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cbc14d570 con 0x7f0cbc14d850 2024-09-15T03:36:28.765 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.759+0000 7f0cc1336700 1 -- 172.21.15.177:0/1864436214 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0cbc14d850 msgr2=0x7f0cbc14dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-15T03:36:28.765 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.759+0000 7f0cc1336700 1 -- 172.21.15.177:0/1864436214 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0cbc14d850 msgr2=0x7f0cbc14dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T03:36:28.765 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.759+0000 7f0cc1336700 1 --2- 172.21.15.177:0/1864436214 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0cbc14d850 0x7f0cbc14dcc0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0cac00b970 tx=0x7f0cac01d070 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T03:36:28.765 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.759+0000 7f0cc1336700 1 --2- 172.21.15.177:0/1864436214 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0cbc14d850 0x7f0cbc14dcc0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0cac00b970 tx=0x7f0cac01d070 comp rx=0 tx=0).stop 2024-09-15T03:36:28.765 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.759+0000 7f0cb1ffb700 1 -- 172.21.15.177:0/1864436214 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0cbc14d850 msgr2=0x7f0cbc14dcc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:36:28.765 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.759+0000 7f0cb1ffb700 1 --2- 172.21.15.177:0/1864436214 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0cbc14d850 0x7f0cbc14dcc0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:28.766 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.759+0000 7f0cb1ffb700 1 --2- 172.21.15.177:0/1864436214 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ca0004ad0 0x7f0ca0006eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:28.766 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.759+0000 7f0cb1ffb700 1 --2- 172.21.15.177:0/1864436214 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f0cbc14e200 0x7f0ca0009a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:28.766 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.759+0000 7f0cb1ffb700 1 --2- 172.21.15.177:0/1864436214 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0ca80074d0 0x7f0ca000c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:28.766 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.759+0000 7f0cb1ffb700 1 -- 172.21.15.177:0/1864436214 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca000ce10 con 0x7f0ca80074d0 2024-09-15T03:36:28.766 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.759+0000 7f0cb1ffb700 1 -- 172.21.15.177:0/1864436214 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca000cf50 con 0x7f0ca0004ad0 2024-09-15T03:36:28.766 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.759+0000 7f0cb1ffb700 1 -- 172.21.15.177:0/1864436214 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca000d0e0 con 0x7f0cbc14e200 2024-09-15T03:36:28.766 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.759+0000 7f0cc1b37700 1 --2- 172.21.15.177:0/1864436214 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ca0004ad0 0x7f0ca0006eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:28.766 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.759+0000 7f0cc1336700 1 --2- 172.21.15.177:0/1864436214 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f0cbc14e200 0x7f0ca0009a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:28.766 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.759+0000 7f0cc1b37700 1 -- 172.21.15.177:0/1864436214 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f0cbc14e200 msgr2=0x7f0ca0009a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:28.766 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.759+0000 7f0cc1b37700 1 --2- 172.21.15.177:0/1864436214 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f0cbc14e200 0x7f0ca0009a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:28.766 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.759+0000 7f0cc1b37700 1 -- 172.21.15.177:0/1864436214 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0ca80074d0 msgr2=0x7f0ca000c490 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T03:36:28.766 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.759+0000 7f0cc1b37700 1 --2- 172.21.15.177:0/1864436214 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0ca80074d0 0x7f0ca000c490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:28.767 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.759+0000 7f0cc1b37700 1 -- 172.21.15.177:0/1864436214 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,mgrmap=0+,monmap=2+}) v3 -- 0x7f0ca000d2a0 con 0x7f0ca0004ad0 2024-09-15T03:36:28.767 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.759+0000 7f0cc1b37700 1 -- 172.21.15.177:0/1864436214 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ca800a250 con 0x7f0ca0004ad0 2024-09-15T03:36:28.767 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.759+0000 7f0cc1336700 1 --2- 172.21.15.177:0/1864436214 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f0cbc14e200 0x7f0ca0009a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T03:36:28.767 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.759+0000 7f0cc1b37700 1 --2- 172.21.15.177:0/1864436214 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ca0004ad0 0x7f0ca0006eb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8011480 tx=0x7f0cb80114b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:28.767 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.763+0000 7f0cb1ffb700 1 -- 172.21.15.177:0/1864436214 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb8011c10 con 0x7f0ca0004ad0 2024-09-15T03:36:28.767 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.763+0000 7f0cb1ffb700 1 -- 172.21.15.177:0/1864436214 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cb8011d70 con 0x7f0ca0004ad0 2024-09-15T03:36:28.767 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.763+0000 7f0cb1ffb700 1 -- 172.21.15.177:0/1864436214 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f0cb8014710 con 0x7f0ca0004ad0 2024-09-15T03:36:28.769 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.767+0000 7f0cb1ffb700 1 -- 172.21.15.177:0/1864436214 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f0cb80149c0 con 0x7f0ca0004ad0 2024-09-15T03:36:28.847 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.847+0000 7f0cc3d9b700 1 -- 172.21.15.177:0/1864436214 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd new", "uuid": "09d93c1d-8ec1-42e8-8b59-5e20958830d6", "id": 0} v 0) v1 -- 0x7f0cbc0716d0 con 0x7f0ca0004ad0 2024-09-15T03:36:28.855 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.851+0000 7f0cb1ffb700 1 -- 172.21.15.177:0/1864436214 <== mon.1 v2:172.21.15.177:3300/0 5 ==== mon_command_ack([{"prefix": "osd new", "uuid": "09d93c1d-8ec1-42e8-8b59-5e20958830d6", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0cb8014d70 con 0x7f0ca0004ad0 2024-09-15T03:36:28.855 INFO:teuthology.orchestra.run.smithi177.stdout:0 2024-09-15T03:36:28.856 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.855+0000 7f0cc3d9b700 1 -- 172.21.15.177:0/1864436214 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ca0004ad0 msgr2=0x7f0ca0006eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:28.856 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.855+0000 7f0cc3d9b700 1 --2- 172.21.15.177:0/1864436214 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ca0004ad0 0x7f0ca0006eb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8011480 tx=0x7f0cb80114b0 comp rx=0 tx=0).stop 2024-09-15T03:36:28.856 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.855+0000 7f0cc3d9b700 1 -- 172.21.15.177:0/1864436214 shutdown_connections 2024-09-15T03:36:28.856 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.855+0000 7f0cc3d9b700 1 --2- 172.21.15.177:0/1864436214 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0ca80074d0 0x7f0ca000c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:28.856 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.855+0000 7f0cc3d9b700 1 --2- 172.21.15.177:0/1864436214 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f0cbc14e200 0x7f0ca0009a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:28.856 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.855+0000 7f0cc3d9b700 1 --2- 172.21.15.177:0/1864436214 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ca0004ad0 0x7f0ca0006eb0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:28.857 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.855+0000 7f0cc3d9b700 1 -- 172.21.15.177:0/1864436214 >> 172.21.15.177:0/1864436214 conn(0x7f0cbc06f140 msgr2=0x7f0cbc139b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:28.857 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.855+0000 7f0cc3d9b700 1 -- 172.21.15.177:0/1864436214 shutdown_connections 2024-09-15T03:36:28.857 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.855+0000 7f0cc3d9b700 1 -- 172.21.15.177:0/1864436214 wait complete. 2024-09-15T03:36:28.868 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd new 378acca0-6784-42c1-a6f8-0ba39a287db3 1 2024-09-15T03:36:28.976 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.975+0000 7ff181035700 1 Processor -- start 2024-09-15T03:36:28.977 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.975+0000 7ff181035700 1 -- start start 2024-09-15T03:36:28.981 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.975+0000 7ff181035700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ff17c0b12d0 0x7ff17c13d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:28.981 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.975+0000 7ff181035700 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff17c0af910 con 0x7ff17c13e360 2024-09-15T03:36:28.981 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.975+0000 7ff181035700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff17c0ad5a0 con 0x7ff17c13df30 2024-09-15T03:36:28.981 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.975+0000 7ff181035700 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff17c0b16a0 con 0x7ff17c0b12d0 2024-09-15T03:36:28.981 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.975+0000 7ff17a59c700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7ff17c13df30 0x7ff17c0aeaf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:37036/0 (socket says 172.21.15.177:37036) 2024-09-15T03:36:28.981 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.975+0000 7ff17a59c700 1 -- 172.21.15.177:0/2198208948 learned_addr learned my addr 172.21.15.177:0/2198208948 (peer_addr_for_me v1:172.21.15.177:0/0) 2024-09-15T03:36:28.981 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.975+0000 7ff17ad9d700 1 --2- 172.21.15.177:0/2198208948 >> v2:172.21.15.110:3300/0 conn(0x7ff17c0b12d0 0x7ff17c13d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:28.981 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.975+0000 7ff179d9b700 1 -- 172.21.15.177:0/2198208948 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1642131509 0 0) 0x7ff17c0ad5a0 con 0x7ff17c13df30 2024-09-15T03:36:28.981 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.975+0000 7ff179d9b700 1 -- 172.21.15.177:0/2198208948 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff164003570 con 0x7ff17c13df30 2024-09-15T03:36:28.982 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.975+0000 7ff179d9b700 1 -- 172.21.15.177:0/2198208948 <== mon.1 v1:172.21.15.177:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3518740921 0 0) 0x7ff164003570 con 0x7ff17c13df30 2024-09-15T03:36:28.982 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.975+0000 7ff179d9b700 1 -- 172.21.15.177:0/2198208948 >> v2:172.21.15.110:3300/0 conn(0x7ff17c0b12d0 msgr2=0x7ff17c13d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:36:28.982 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.975+0000 7ff179d9b700 1 --2- 172.21.15.177:0/2198208948 >> v2:172.21.15.110:3300/0 conn(0x7ff17c0b12d0 0x7ff17c13d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:28.982 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.975+0000 7ff179d9b700 1 -- 172.21.15.177:0/2198208948 >> v1:172.21.15.110:6789/0 conn(0x7ff17c13e360 legacy=0x7ff17c0af200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:28.982 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.975+0000 7ff179d9b700 1 -- 172.21.15.177:0/2198208948 --> v1:172.21.15.177:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff17c0ae3b0 con 0x7ff17c13df30 2024-09-15T03:36:28.982 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.975+0000 7ff179d9b700 1 -- 172.21.15.177:0/2198208948 <== mon.1 v1:172.21.15.177:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 644009436 0 0) 0x7ff16c001470 con 0x7ff17c13df30 2024-09-15T03:36:28.982 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.975+0000 7ff179d9b700 1 -- 172.21.15.177:0/2198208948 >> v1:172.21.15.177:6789/0 conn(0x7ff17c13df30 legacy=0x7ff17c0aeaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:28.982 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.975+0000 7ff179d9b700 1 --2- 172.21.15.177:0/2198208948 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff164003a60 0x7ff164003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:28.983 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.975+0000 7ff179d9b700 1 --2- 172.21.15.177:0/2198208948 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7ff164004810 0x7ff164004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:28.983 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.975+0000 7ff179d9b700 1 --2- 172.21.15.177:0/2198208948 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1640051c0 0x7ff16400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:28.983 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.975+0000 7ff179d9b700 1 -- 172.21.15.177:0/2198208948 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff17c0b16a0 con 0x7ff164003a60 2024-09-15T03:36:28.983 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.979+0000 7ff179d9b700 1 -- 172.21.15.177:0/2198208948 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16400c0d0 con 0x7ff1640051c0 2024-09-15T03:36:28.983 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.979+0000 7ff179d9b700 1 -- 172.21.15.177:0/2198208948 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff16400c290 con 0x7ff164004810 2024-09-15T03:36:28.983 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.979+0000 7ff17b59e700 1 --2- 172.21.15.177:0/2198208948 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1640051c0 0x7ff16400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:28.983 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.979+0000 7ff17ad9d700 1 --2- 172.21.15.177:0/2198208948 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff164003a60 0x7ff164003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:28.983 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.979+0000 7ff17a59c700 1 --2- 172.21.15.177:0/2198208948 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7ff164004810 0x7ff164004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:28.983 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.979+0000 7ff17b59e700 1 -- 172.21.15.177:0/2198208948 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7ff164004810 msgr2=0x7ff164004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:36:28.983 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.979+0000 7ff17b59e700 1 --2- 172.21.15.177:0/2198208948 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7ff164004810 0x7ff164004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:28.983 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.979+0000 7ff17b59e700 1 -- 172.21.15.177:0/2198208948 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff164003a60 msgr2=0x7ff164003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:36:28.983 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.979+0000 7ff17b59e700 1 --2- 172.21.15.177:0/2198208948 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff164003a60 0x7ff164003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:28.984 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.979+0000 7ff17b59e700 1 -- 172.21.15.177:0/2198208948 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff164003570 con 0x7ff1640051c0 2024-09-15T03:36:28.984 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.979+0000 7ff17ad9d700 1 --2- 172.21.15.177:0/2198208948 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff164003a60 0x7ff164003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T03:36:28.984 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.979+0000 7ff17b59e700 1 -- 172.21.15.177:0/2198208948 reap_dead start 2024-09-15T03:36:28.984 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.979+0000 7ff17b59e700 1 --2- 172.21.15.177:0/2198208948 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1640051c0 0x7ff16400ba60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff174001fc0 tx=0x7ff174001640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4384982fde359815 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:28.984 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.979+0000 7ff179d9b700 1 -- 172.21.15.177:0/2198208948 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff174001950 con 0x7ff1640051c0 2024-09-15T03:36:28.984 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.979+0000 7ff179d9b700 1 -- 172.21.15.177:0/2198208948 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff17400c8b0 con 0x7ff1640051c0 2024-09-15T03:36:28.984 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.979+0000 7ff179d9b700 1 -- 172.21.15.177:0/2198208948 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17400cc50 con 0x7ff1640051c0 2024-09-15T03:36:28.984 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.979+0000 7ff181035700 1 -- 172.21.15.177:0/2198208948 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1640051c0 msgr2=0x7ff16400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:28.984 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.979+0000 7ff181035700 1 --2- 172.21.15.177:0/2198208948 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1640051c0 0x7ff16400ba60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff174001fc0 tx=0x7ff174001640 comp rx=0 tx=0).stop 2024-09-15T03:36:28.984 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.979+0000 7ff181035700 1 -- 172.21.15.177:0/2198208948 shutdown_connections 2024-09-15T03:36:28.984 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.979+0000 7ff181035700 1 --2- 172.21.15.177:0/2198208948 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1640051c0 0x7ff16400ba60 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:28.984 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.979+0000 7ff181035700 1 -- 172.21.15.177:0/2198208948 >> 172.21.15.177:0/2198208948 conn(0x7ff17c06ef40 msgr2=0x7ff17c06f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:28.985 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.979+0000 7ff181035700 1 -- 172.21.15.177:0/2198208948 shutdown_connections 2024-09-15T03:36:28.985 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.979+0000 7ff181035700 1 -- 172.21.15.177:0/2198208948 wait complete. 2024-09-15T03:36:28.988 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.983+0000 7ff181035700 1 Processor -- start 2024-09-15T03:36:28.988 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.983+0000 7ff181035700 1 -- start start 2024-09-15T03:36:28.988 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.983+0000 7ff181035700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7ff1640051c0 0x7ff17c139b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:28.988 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.983+0000 7ff181035700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff17c13b640 0x7ff17c0aa470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:28.988 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.983+0000 7ff181035700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff17c0a7fe0 0x7ff17c0a8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:28.988 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.983+0000 7ff181035700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff17c0aa9b0 con 0x7ff17c0a7fe0 2024-09-15T03:36:28.988 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.983+0000 7ff181035700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff17c0a9de0 con 0x7ff17c13b640 2024-09-15T03:36:28.989 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.983+0000 7ff181035700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff17c0aa050 con 0x7ff1640051c0 2024-09-15T03:36:28.989 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.983+0000 7ff17a59c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff17c13b640 0x7ff17c0aa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:28.989 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.983+0000 7ff17ad9d700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7ff1640051c0 0x7ff17c139b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:28.989 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.983+0000 7ff17a59c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff17c13b640 0x7ff17c0aa470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:54076/0 (socket says 172.21.15.177:54076) 2024-09-15T03:36:28.989 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.983+0000 7ff17a59c700 1 -- 172.21.15.177:0/4019902796 learned_addr learned my addr 172.21.15.177:0/4019902796 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-15T03:36:28.989 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.983+0000 7ff17b59e700 1 --2- 172.21.15.177:0/4019902796 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff17c0a7fe0 0x7ff17c0a8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:28.989 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.983+0000 7ff17a59c700 1 -- 172.21.15.177:0/4019902796 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7ff1640051c0 msgr2=0x7ff17c139b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:28.989 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.983+0000 7ff17a59c700 1 --2- 172.21.15.177:0/4019902796 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7ff1640051c0 0x7ff17c139b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:28.989 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.983+0000 7ff17a59c700 1 -- 172.21.15.177:0/4019902796 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff17c0a7fe0 msgr2=0x7ff17c0a8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:28.989 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.983+0000 7ff17a59c700 1 --2- 172.21.15.177:0/4019902796 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff17c0a7fe0 0x7ff17c0a8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:28.989 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.983+0000 7ff17a59c700 1 -- 172.21.15.177:0/4019902796 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff17c13ca10 con 0x7ff17c13b640 2024-09-15T03:36:28.989 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.983+0000 7ff17a59c700 1 --2- 172.21.15.177:0/4019902796 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff17c13b640 0x7ff17c0aa470 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff16c00e8b0 tx=0x7ff16c000d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:28.990 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.983+0000 7ff162ffd700 1 -- 172.21.15.177:0/4019902796 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff16c012500 con 0x7ff17c13b640 2024-09-15T03:36:28.990 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.983+0000 7ff162ffd700 1 -- 172.21.15.177:0/4019902796 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff16c010040 con 0x7ff17c13b640 2024-09-15T03:36:28.990 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.983+0000 7ff181035700 1 -- 172.21.15.177:0/4019902796 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff17c13bcc0 con 0x7ff17c13b640 2024-09-15T03:36:28.990 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.983+0000 7ff162ffd700 1 -- 172.21.15.177:0/4019902796 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff16c0128d0 con 0x7ff17c13b640 2024-09-15T03:36:28.990 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.983+0000 7ff181035700 1 -- 172.21.15.177:0/4019902796 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff17c13c190 con 0x7ff17c13b640 2024-09-15T03:36:28.990 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.987+0000 7ff162ffd700 1 -- 172.21.15.177:0/4019902796 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7ff16c012c40 con 0x7ff17c13b640 2024-09-15T03:36:28.990 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.987+0000 7ff162ffd700 1 -- 172.21.15.177:0/4019902796 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 836+0+0 (secure 0 0 0) 0x7ff16c01eac0 con 0x7ff17c13b640 2024-09-15T03:36:28.990 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.987+0000 7ff181035700 1 -- 172.21.15.177:0/4019902796 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff17c064c70 con 0x7ff17c13b640 2024-09-15T03:36:28.992 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:28.991+0000 7ff162ffd700 1 -- 172.21.15.177:0/4019902796 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7ff16c019070 con 0x7ff17c13b640 2024-09-15T03:36:29.067 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.063+0000 7ff181035700 1 -- 172.21.15.177:0/4019902796 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd new", "uuid": "378acca0-6784-42c1-a6f8-0ba39a287db3", "id": 1} v 0) v1 -- 0x7ff17c13d640 con 0x7ff17c13b640 2024-09-15T03:36:29.077 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.075+0000 7ff162ffd700 1 -- 172.21.15.177:0/4019902796 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "378acca0-6784-42c1-a6f8-0ba39a287db3", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff16c01e490 con 0x7ff17c13b640 2024-09-15T03:36:29.077 INFO:teuthology.orchestra.run.smithi177.stdout:1 2024-09-15T03:36:29.078 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.075+0000 7ff181035700 1 -- 172.21.15.177:0/4019902796 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff17c13b640 msgr2=0x7ff17c0aa470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:29.078 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.075+0000 7ff181035700 1 --2- 172.21.15.177:0/4019902796 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff17c13b640 0x7ff17c0aa470 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff16c00e8b0 tx=0x7ff16c000d70 comp rx=0 tx=0).stop 2024-09-15T03:36:29.078 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.075+0000 7ff181035700 1 -- 172.21.15.177:0/4019902796 shutdown_connections 2024-09-15T03:36:29.078 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.075+0000 7ff181035700 1 --2- 172.21.15.177:0/4019902796 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff17c0a7fe0 0x7ff17c0a8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:29.078 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.075+0000 7ff181035700 1 --2- 172.21.15.177:0/4019902796 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff17c13b640 0x7ff17c0aa470 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:29.079 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.075+0000 7ff181035700 1 --2- 172.21.15.177:0/4019902796 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7ff1640051c0 0x7ff17c139b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:29.079 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.075+0000 7ff181035700 1 -- 172.21.15.177:0/4019902796 >> 172.21.15.177:0/4019902796 conn(0x7ff17c06ef40 msgr2=0x7ff17c137230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:29.079 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.075+0000 7ff181035700 1 -- 172.21.15.177:0/4019902796 shutdown_connections 2024-09-15T03:36:29.079 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.075+0000 7ff181035700 1 -- 172.21.15.177:0/4019902796 wait complete. 2024-09-15T03:36:29.090 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd new b1c2c060-985a-429c-bf05-ccfd5fdf3d8f 2 2024-09-15T03:36:29.197 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.195+0000 7f8cfd5dc700 1 Processor -- start 2024-09-15T03:36:29.197 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.195+0000 7f8cfd5dc700 1 -- start start 2024-09-15T03:36:29.198 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.195+0000 7f8cfd5dc700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f8cf80acf70 0x7f8cf8142290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:29.198 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.195+0000 7f8cfd5dc700 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cf8141ca0 con 0x7f8cf81427d0 2024-09-15T03:36:29.198 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.195+0000 7f8cfd5dc700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cf80ad340 con 0x7f8cf806c2b0 2024-09-15T03:36:29.198 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.195+0000 7f8cfd5dc700 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cf806ca60 con 0x7f8cf80acf70 2024-09-15T03:36:29.198 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.195+0000 7f8cf77fe700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f8cf806c2b0 0x7f8cf81455a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:37042/0 (socket says 172.21.15.177:37042) 2024-09-15T03:36:29.198 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.195+0000 7f8cf77fe700 1 -- 172.21.15.177:0/1762812765 learned_addr learned my addr 172.21.15.177:0/1762812765 (peer_addr_for_me v1:172.21.15.177:0/0) 2024-09-15T03:36:29.198 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.195+0000 7f8cf7fff700 1 --2- 172.21.15.177:0/1762812765 >> v2:172.21.15.110:3300/0 conn(0x7f8cf80acf70 0x7f8cf8142290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:29.199 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.195+0000 7f8cf6ffd700 1 -- 172.21.15.177:0/1762812765 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 40087577 0 0) 0x7f8cf80ad340 con 0x7f8cf806c2b0 2024-09-15T03:36:29.200 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.195+0000 7f8cf6ffd700 1 -- 172.21.15.177:0/1762812765 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ce4003570 con 0x7f8cf806c2b0 2024-09-15T03:36:29.200 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.195+0000 7f8cf6ffd700 1 -- 172.21.15.177:0/1762812765 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 993003379 0 0) 0x7f8cf8141ca0 con 0x7f8cf81427d0 2024-09-15T03:36:29.200 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.195+0000 7f8cf6ffd700 1 -- 172.21.15.177:0/1762812765 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8cf80ad340 con 0x7f8cf81427d0 2024-09-15T03:36:29.200 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.195+0000 7f8cf6ffd700 1 -- 172.21.15.177:0/1762812765 <== mon.1 v1:172.21.15.177:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1040049944 0 0) 0x7f8ce4003570 con 0x7f8cf806c2b0 2024-09-15T03:36:29.200 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.195+0000 7f8cf7fff700 1 -- 172.21.15.177:0/1762812765 >> v1:172.21.15.177:6789/0 conn(0x7f8cf806c2b0 legacy=0x7f8cf81455a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:29.200 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.199+0000 7f8cf7fff700 1 -- 172.21.15.177:0/1762812765 >> v1:172.21.15.110:6789/0 conn(0x7f8cf81427d0 legacy=0x7f8cf814c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:29.200 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.199+0000 7f8cf7fff700 1 -- 172.21.15.177:0/1762812765 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cf806d950 con 0x7f8cf80acf70 2024-09-15T03:36:29.200 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.199+0000 7f8cf7fff700 1 --2- 172.21.15.177:0/1762812765 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8cf80acf70 0x7f8cf8142290 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8ce0009910 tx=0x7f8ce00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d7b2f0a8eafa0ab server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:29.200 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.199+0000 7f8cf6ffd700 1 -- 172.21.15.177:0/1762812765 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf00013f0 con 0x7f8cf80acf70 2024-09-15T03:36:29.200 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.199+0000 7f8cf6ffd700 1 -- 172.21.15.177:0/1762812765 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ce0011420 con 0x7f8cf80acf70 2024-09-15T03:36:29.200 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.199+0000 7f8cf6ffd700 1 -- 172.21.15.177:0/1762812765 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce80017f0 con 0x7f8cf80acf70 2024-09-15T03:36:29.206 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.199+0000 7f8cfd5dc700 1 -- 172.21.15.177:0/1762812765 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8cf80acf70 msgr2=0x7f8cf8142290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:29.206 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.199+0000 7f8cfd5dc700 1 --2- 172.21.15.177:0/1762812765 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8cf80acf70 0x7f8cf8142290 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8ce0009910 tx=0x7f8ce00091a0 comp rx=0 tx=0).stop 2024-09-15T03:36:29.206 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.199+0000 7f8cfd5dc700 1 -- 172.21.15.177:0/1762812765 shutdown_connections 2024-09-15T03:36:29.206 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.199+0000 7f8cfd5dc700 1 --2- 172.21.15.177:0/1762812765 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8cf80acf70 0x7f8cf8142290 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:29.206 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.199+0000 7f8cfd5dc700 1 -- 172.21.15.177:0/1762812765 >> 172.21.15.177:0/1762812765 conn(0x7f8cf8067a10 msgr2=0x7f8cf8067e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:29.206 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.199+0000 7f8cfd5dc700 1 -- 172.21.15.177:0/1762812765 shutdown_connections 2024-09-15T03:36:29.206 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.199+0000 7f8cfd5dc700 1 -- 172.21.15.177:0/1762812765 wait complete. 2024-09-15T03:36:29.206 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.199+0000 7f8cfd5dc700 1 Processor -- start 2024-09-15T03:36:29.206 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.199+0000 7f8cfd5dc700 1 -- start start 2024-09-15T03:36:29.206 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.199+0000 7f8cfd5dc700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8cf806c2b0 0x7f8cf8155390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:29.207 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.199+0000 7f8cfd5dc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8cf80acf70 0x7f8cf81400d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:29.207 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.199+0000 7f8cfd5dc700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f8cf81427d0 0x7f8cf8140610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:29.207 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.199+0000 7f8cfd5dc700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf8140b50 con 0x7f8cf80acf70 2024-09-15T03:36:29.207 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.199+0000 7f8cfd5dc700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf8155a70 con 0x7f8cf806c2b0 2024-09-15T03:36:29.207 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.199+0000 7f8cfd5dc700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf80a7480 con 0x7f8cf81427d0 2024-09-15T03:36:29.207 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.203+0000 7f8cf7fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8cf806c2b0 0x7f8cf8155390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:29.207 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.203+0000 7f8cf7fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8cf806c2b0 0x7f8cf8155390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:54080/0 (socket says 172.21.15.177:54080) 2024-09-15T03:36:29.207 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.203+0000 7f8cf7fff700 1 -- 172.21.15.177:0/3623565227 learned_addr learned my addr 172.21.15.177:0/3623565227 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-15T03:36:29.207 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.203+0000 7f8cf77fe700 1 --2- 172.21.15.177:0/3623565227 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8cf80acf70 0x7f8cf81400d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:29.207 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.203+0000 7f8cfcddb700 1 --2- 172.21.15.177:0/3623565227 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f8cf81427d0 0x7f8cf8140610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:29.207 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.203+0000 7f8cf7fff700 1 -- 172.21.15.177:0/3623565227 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f8cf81427d0 msgr2=0x7f8cf8140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:29.207 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.203+0000 7f8cf7fff700 1 --2- 172.21.15.177:0/3623565227 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f8cf81427d0 0x7f8cf8140610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:29.207 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.203+0000 7f8cf7fff700 1 -- 172.21.15.177:0/3623565227 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8cf80acf70 msgr2=0x7f8cf81400d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:29.208 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.203+0000 7f8cf7fff700 1 --2- 172.21.15.177:0/3623565227 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8cf80acf70 0x7f8cf81400d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:29.208 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.203+0000 7f8cf7fff700 1 -- 172.21.15.177:0/3623565227 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ce0009420 con 0x7f8cf806c2b0 2024-09-15T03:36:29.208 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.203+0000 7f8cf77fe700 1 --2- 172.21.15.177:0/3623565227 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8cf80acf70 0x7f8cf81400d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T03:36:29.208 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.203+0000 7f8cfcddb700 1 --2- 172.21.15.177:0/3623565227 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f8cf81427d0 0x7f8cf8140610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T03:36:29.208 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.203+0000 7f8cf7fff700 1 --2- 172.21.15.177:0/3623565227 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8cf806c2b0 0x7f8cf8155390 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8cf806a820 tx=0x7f8ce0017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:29.208 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.203+0000 7f8ceffff700 1 -- 172.21.15.177:0/3623565227 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce0010a80 con 0x7f8cf806c2b0 2024-09-15T03:36:29.208 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.203+0000 7f8ceffff700 1 -- 172.21.15.177:0/3623565227 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ce0010be0 con 0x7f8cf806c2b0 2024-09-15T03:36:29.208 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.203+0000 7f8cfd5dc700 1 -- 172.21.15.177:0/3623565227 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cf8141de0 con 0x7f8cf806c2b0 2024-09-15T03:36:29.208 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.203+0000 7f8ceffff700 1 -- 172.21.15.177:0/3623565227 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce001d430 con 0x7f8cf806c2b0 2024-09-15T03:36:29.208 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.203+0000 7f8cfd5dc700 1 -- 172.21.15.177:0/3623565227 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cf813d270 con 0x7f8cf806c2b0 2024-09-15T03:36:29.208 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.203+0000 7f8ceffff700 1 -- 172.21.15.177:0/3623565227 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f8ce001d820 con 0x7f8cf806c2b0 2024-09-15T03:36:29.208 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.203+0000 7f8ceffff700 1 -- 172.21.15.177:0/3623565227 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 951+0+0 (secure 0 0 0) 0x7f8ce0026880 con 0x7f8cf806c2b0 2024-09-15T03:36:29.208 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.203+0000 7f8cfd5dc700 1 -- 172.21.15.177:0/3623565227 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cf80716d0 con 0x7f8cf806c2b0 2024-09-15T03:36:29.209 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.207+0000 7f8ceffff700 1 -- 172.21.15.177:0/3623565227 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f8ce002b060 con 0x7f8cf806c2b0 2024-09-15T03:36:29.285 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.283+0000 7f8cfd5dc700 1 -- 172.21.15.177:0/3623565227 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b1c2c060-985a-429c-bf05-ccfd5fdf3d8f", "id": 2} v 0) v1 -- 0x7f8cf806d390 con 0x7f8cf806c2b0 2024-09-15T03:36:29.295 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.291+0000 7f8ceffff700 1 -- 172.21.15.177:0/3623565227 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b1c2c060-985a-429c-bf05-ccfd5fdf3d8f", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f8ce0021070 con 0x7f8cf806c2b0 2024-09-15T03:36:29.295 INFO:teuthology.orchestra.run.smithi177.stdout:2 2024-09-15T03:36:29.296 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.295+0000 7f8cfd5dc700 1 -- 172.21.15.177:0/3623565227 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8cf806c2b0 msgr2=0x7f8cf8155390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:29.296 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.295+0000 7f8cfd5dc700 1 --2- 172.21.15.177:0/3623565227 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8cf806c2b0 0x7f8cf8155390 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8cf806a820 tx=0x7f8ce0017000 comp rx=0 tx=0).stop 2024-09-15T03:36:29.297 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.295+0000 7f8cfd5dc700 1 -- 172.21.15.177:0/3623565227 shutdown_connections 2024-09-15T03:36:29.297 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.295+0000 7f8cfd5dc700 1 --2- 172.21.15.177:0/3623565227 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f8cf81427d0 0x7f8cf8140610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:29.297 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.295+0000 7f8cfd5dc700 1 --2- 172.21.15.177:0/3623565227 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8cf80acf70 0x7f8cf81400d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:29.297 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.295+0000 7f8cfd5dc700 1 --2- 172.21.15.177:0/3623565227 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8cf806c2b0 0x7f8cf8155390 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:29.297 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.295+0000 7f8cfd5dc700 1 -- 172.21.15.177:0/3623565227 >> 172.21.15.177:0/3623565227 conn(0x7f8cf8067a10 msgr2=0x7f8cf81455a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:29.297 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.295+0000 7f8cfd5dc700 1 -- 172.21.15.177:0/3623565227 shutdown_connections 2024-09-15T03:36:29.297 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.295+0000 7f8cfd5dc700 1 -- 172.21.15.177:0/3623565227 wait complete. 2024-09-15T03:36:29.310 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd new aca55104-d76b-4719-8837-252c2e55d0cd 3 2024-09-15T03:36:29.336 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-15T03:36:29.331+0000 7fa4cdfbb000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-15T03:36:29.393 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-15T03:36:29.387+0000 7fa4cdfbb000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-15T03:36:29.416 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.411+0000 7f5496da0700 1 Processor -- start 2024-09-15T03:36:29.417 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.411+0000 7f5496da0700 1 -- start start 2024-09-15T03:36:29.417 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.411+0000 7f5496da0700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f54900aa970 0x7f549006e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:29.417 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.411+0000 7f5496da0700 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f549006e160 con 0x7f549006c350 2024-09-15T03:36:29.417 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.411+0000 7f5496da0700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54900aad40 con 0x7f549006cd00 2024-09-15T03:36:29.417 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.411+0000 7f5496da0700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54901377a0 con 0x7f54900aa970 2024-09-15T03:36:29.417 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.411+0000 7f5494b3c700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f54900aa970 0x7f549006e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:29.418 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.411+0000 7f549533d700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f549006cd00 0x7f5490137090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:37050/0 (socket says 172.21.15.177:37050) 2024-09-15T03:36:29.418 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.411+0000 7f549533d700 1 -- 172.21.15.177:0/808597526 learned_addr learned my addr 172.21.15.177:0/808597526 (peer_addr_for_me v1:172.21.15.177:0/0) 2024-09-15T03:36:29.418 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.411+0000 7f548f7fe700 1 -- 172.21.15.177:0/808597526 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3326489783 0 0) 0x7f54900aad40 con 0x7f549006cd00 2024-09-15T03:36:29.418 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.411+0000 7f548f7fe700 1 -- 172.21.15.177:0/808597526 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5478003570 con 0x7f549006cd00 2024-09-15T03:36:29.418 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.411+0000 7f5494b3c700 1 -- 172.21.15.177:0/808597526 >> v1:172.21.15.177:6789/0 conn(0x7f549006cd00 legacy=0x7f5490137090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:29.419 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.411+0000 7f5494b3c700 1 -- 172.21.15.177:0/808597526 >> v1:172.21.15.110:6789/0 conn(0x7f549006c350 legacy=0x7f5490136980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:29.419 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.411+0000 7f5494b3c700 1 -- 172.21.15.177:0/808597526 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54900acec0 con 0x7f54900aa970 2024-09-15T03:36:29.420 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.411+0000 7f5494b3c700 1 --2- 172.21.15.177:0/808597526 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f54900aa970 0x7f549006e510 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f54800098e0 tx=0x7f54800091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=916f245b69bdb5ac server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:29.420 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.411+0000 7f548f7fe700 1 -- 172.21.15.177:0/808597526 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548000b640 con 0x7f54900aa970 2024-09-15T03:36:29.420 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.411+0000 7f548f7fe700 1 -- 172.21.15.177:0/808597526 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f548000b7a0 con 0x7f54900aa970 2024-09-15T03:36:29.420 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.411+0000 7f548f7fe700 1 -- 172.21.15.177:0/808597526 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548000bb40 con 0x7f54900aa970 2024-09-15T03:36:29.420 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.411+0000 7f5496da0700 1 -- 172.21.15.177:0/808597526 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f54900aa970 msgr2=0x7f549006e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:29.421 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.411+0000 7f5496da0700 1 --2- 172.21.15.177:0/808597526 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f54900aa970 0x7f549006e510 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f54800098e0 tx=0x7f54800091a0 comp rx=0 tx=0).stop 2024-09-15T03:36:29.421 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.411+0000 7f5496da0700 1 -- 172.21.15.177:0/808597526 shutdown_connections 2024-09-15T03:36:29.421 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.411+0000 7f5496da0700 1 --2- 172.21.15.177:0/808597526 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f54900aa970 0x7f549006e510 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:29.421 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.411+0000 7f5496da0700 1 -- 172.21.15.177:0/808597526 >> 172.21.15.177:0/808597526 conn(0x7f54900b0a70 msgr2=0x7f54900b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:29.421 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.411+0000 7f5496da0700 1 -- 172.21.15.177:0/808597526 shutdown_connections 2024-09-15T03:36:29.421 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.411+0000 7f5496da0700 1 -- 172.21.15.177:0/808597526 wait complete. 2024-09-15T03:36:29.422 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.415+0000 7f5496da0700 1 Processor -- start 2024-09-15T03:36:29.422 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.415+0000 7f5496da0700 1 -- start start 2024-09-15T03:36:29.427 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.415+0000 7f5496da0700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f549006c350 0x7f5490146a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:29.427 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.415+0000 7f5496da0700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f549006cd00 0x7f549014d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:29.427 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.415+0000 7f5496da0700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54900aa970 0x7f549013b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:29.428 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.415+0000 7f5496da0700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549013db00 con 0x7f54900aa970 2024-09-15T03:36:29.428 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.415+0000 7f5496da0700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549014d4d0 con 0x7f549006cd00 2024-09-15T03:36:29.428 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.415+0000 7f5496da0700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f549013b890 con 0x7f549006c350 2024-09-15T03:36:29.428 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.415+0000 7f548ffff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f549006cd00 0x7f549014d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:29.428 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.415+0000 7f548ffff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f549006cd00 0x7f549014d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:54110/0 (socket says 172.21.15.177:54110) 2024-09-15T03:36:29.429 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.415+0000 7f548ffff700 1 -- 172.21.15.177:0/1088845009 learned_addr learned my addr 172.21.15.177:0/1088845009 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-15T03:36:29.429 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.415+0000 7f5494b3c700 1 --2- 172.21.15.177:0/1088845009 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f549006c350 0x7f5490146a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:29.429 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.415+0000 7f549533d700 1 --2- 172.21.15.177:0/1088845009 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54900aa970 0x7f549013b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:29.429 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.415+0000 7f548ffff700 1 -- 172.21.15.177:0/1088845009 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f549006c350 msgr2=0x7f5490146a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:29.429 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.415+0000 7f548ffff700 1 --2- 172.21.15.177:0/1088845009 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f549006c350 0x7f5490146a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:29.429 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.415+0000 7f548ffff700 1 -- 172.21.15.177:0/1088845009 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54900aa970 msgr2=0x7f549013b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:29.430 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.415+0000 7f548ffff700 1 --2- 172.21.15.177:0/1088845009 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54900aa970 0x7f549013b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:29.430 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.415+0000 7f548ffff700 1 -- 172.21.15.177:0/1088845009 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5480009420 con 0x7f549006cd00 2024-09-15T03:36:29.430 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.415+0000 7f548ffff700 1 --2- 172.21.15.177:0/1088845009 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f549006cd00 0x7f549014d9f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f548400c5c0 tx=0x7f548400c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:29.430 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.415+0000 7f548cff9700 1 -- 172.21.15.177:0/1088845009 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5484011590 con 0x7f549006cd00 2024-09-15T03:36:29.430 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.415+0000 7f548cff9700 1 -- 172.21.15.177:0/1088845009 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5484011710 con 0x7f549006cd00 2024-09-15T03:36:29.430 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.415+0000 7f5496da0700 1 -- 172.21.15.177:0/1088845009 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f549013deb0 con 0x7f549006cd00 2024-09-15T03:36:29.431 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.415+0000 7f548cff9700 1 -- 172.21.15.177:0/1088845009 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5484011ad0 con 0x7f549006cd00 2024-09-15T03:36:29.431 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.419+0000 7f5496da0700 1 -- 172.21.15.177:0/1088845009 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f549014cfb0 con 0x7f549006cd00 2024-09-15T03:36:29.431 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.419+0000 7f548cff9700 1 -- 172.21.15.177:0/1088845009 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f548401c440 con 0x7f549006cd00 2024-09-15T03:36:29.431 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.419+0000 7f548cff9700 1 -- 172.21.15.177:0/1088845009 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1066+0+0 (secure 0 0 0) 0x7f5484012420 con 0x7f549006cd00 2024-09-15T03:36:29.431 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.419+0000 7f5496da0700 1 -- 172.21.15.177:0/1088845009 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f549014d6e0 con 0x7f549006cd00 2024-09-15T03:36:29.431 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.419+0000 7f548cff9700 1 -- 172.21.15.177:0/1088845009 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f5484017070 con 0x7f549006cd00 2024-09-15T03:36:29.457 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-15T03:36:29.455+0000 7febd9357000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-15T03:36:29.502 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-15T03:36:29.499+0000 7fa4cdfbb000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-15T03:36:29.517 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.515+0000 7f5496da0700 1 -- 172.21.15.177:0/1088845009 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd new", "uuid": "aca55104-d76b-4719-8837-252c2e55d0cd", "id": 3} v 0) v1 -- 0x7f54900716d0 con 0x7f549006cd00 2024-09-15T03:36:29.524 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.523+0000 7f548cff9700 1 -- 172.21.15.177:0/1088845009 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "aca55104-d76b-4719-8837-252c2e55d0cd", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f548401c6f0 con 0x7f549006cd00 2024-09-15T03:36:29.526 INFO:teuthology.orchestra.run.smithi177.stdout:3 2024-09-15T03:36:29.526 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.523+0000 7f5496da0700 1 -- 172.21.15.177:0/1088845009 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f549006cd00 msgr2=0x7f549014d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:29.527 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.523+0000 7f5496da0700 1 --2- 172.21.15.177:0/1088845009 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f549006cd00 0x7f549014d9f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f548400c5c0 tx=0x7f548400c8d0 comp rx=0 tx=0).stop 2024-09-15T03:36:29.527 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.523+0000 7f5496da0700 1 -- 172.21.15.177:0/1088845009 shutdown_connections 2024-09-15T03:36:29.527 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.523+0000 7f5496da0700 1 --2- 172.21.15.177:0/1088845009 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54900aa970 0x7f549013b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:29.527 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.523+0000 7f5496da0700 1 --2- 172.21.15.177:0/1088845009 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f549006cd00 0x7f549014d9f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:29.527 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.523+0000 7f5496da0700 1 --2- 172.21.15.177:0/1088845009 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f549006c350 0x7f5490146a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:29.527 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.523+0000 7f5496da0700 1 -- 172.21.15.177:0/1088845009 >> 172.21.15.177:0/1088845009 conn(0x7f54900b0a70 msgr2=0x7f5490133fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:29.527 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.523+0000 7f5496da0700 1 -- 172.21.15.177:0/1088845009 shutdown_connections 2024-09-15T03:36:29.527 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.523+0000 7f5496da0700 1 -- 172.21.15.177:0/1088845009 wait complete. 2024-09-15T03:36:29.538 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd new 7a87be3c-b6ff-43a4-8672-39d3006a0e2b 4 2024-09-15T03:36:29.638 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.635+0000 7fa9da4a2700 1 Processor -- start 2024-09-15T03:36:29.639 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.635+0000 7fa9da4a2700 1 -- start start 2024-09-15T03:36:29.643 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.639+0000 7fa9da4a2700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa9d406f420 0x7fa9d413d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:29.643 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.639+0000 7fa9da4a2700 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9d406d4b0 con 0x7fa9d406d940 2024-09-15T03:36:29.644 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.639+0000 7fa9da4a2700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9d406f7f0 con 0x7fa9d413dde0 2024-09-15T03:36:29.644 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.639+0000 7fa9da4a2700 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9d40ac5d0 con 0x7fa9d406f420 2024-09-15T03:36:29.644 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.639+0000 7fa9d8c9f700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fa9d413dde0 0x7fa9d40ab7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:37052/0 (socket says 172.21.15.177:37052) 2024-09-15T03:36:29.644 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.639+0000 7fa9d8c9f700 1 -- 172.21.15.177:0/3817933500 learned_addr learned my addr 172.21.15.177:0/3817933500 (peer_addr_for_me v1:172.21.15.177:0/0) 2024-09-15T03:36:29.645 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.639+0000 7fa9d94a0700 1 --2- 172.21.15.177:0/3817933500 >> v2:172.21.15.110:3300/0 conn(0x7fa9d406f420 0x7fa9d413d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:29.645 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.639+0000 7fa9cbfff700 1 -- 172.21.15.177:0/3817933500 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 82636268 0 0) 0x7fa9d406f7f0 con 0x7fa9d413dde0 2024-09-15T03:36:29.645 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.639+0000 7fa9cbfff700 1 -- 172.21.15.177:0/3817933500 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9c0003570 con 0x7fa9d413dde0 2024-09-15T03:36:29.645 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.639+0000 7fa9cbfff700 1 -- 172.21.15.177:0/3817933500 <== mon.1 v1:172.21.15.177:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 207955952 0 0) 0x7fa9c0003570 con 0x7fa9d413dde0 2024-09-15T03:36:29.645 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.639+0000 7fa9cbfff700 1 -- 172.21.15.177:0/3817933500 >> v2:172.21.15.110:3300/0 conn(0x7fa9d406f420 msgr2=0x7fa9d413d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:36:29.645 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.639+0000 7fa9cbfff700 1 --2- 172.21.15.177:0/3817933500 >> v2:172.21.15.110:3300/0 conn(0x7fa9d406f420 0x7fa9d413d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:29.645 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.639+0000 7fa9cbfff700 1 -- 172.21.15.177:0/3817933500 >> v1:172.21.15.110:6789/0 conn(0x7fa9d406d940 legacy=0x7fa9d40abec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:29.645 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.639+0000 7fa9cbfff700 1 -- 172.21.15.177:0/3817933500 --> v1:172.21.15.177:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d40ad640 con 0x7fa9d413dde0 2024-09-15T03:36:29.645 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.639+0000 7fa9cbfff700 1 -- 172.21.15.177:0/3817933500 <== mon.1 v1:172.21.15.177:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 644009436 0 0) 0x7fa9c4001470 con 0x7fa9d413dde0 2024-09-15T03:36:29.645 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.639+0000 7fa9cbfff700 1 -- 172.21.15.177:0/3817933500 >> v1:172.21.15.177:6789/0 conn(0x7fa9d413dde0 legacy=0x7fa9d40ab7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:29.646 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.639+0000 7fa9cbfff700 1 --2- 172.21.15.177:0/3817933500 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fa9c0003a60 0x7fa9c0003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:29.646 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.639+0000 7fa9cbfff700 1 --2- 172.21.15.177:0/3817933500 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa9c0004810 0x7fa9c0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:29.646 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.639+0000 7fa9cbfff700 1 --2- 172.21.15.177:0/3817933500 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa9c00051c0 0x7fa9c000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:29.646 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.639+0000 7fa9cbfff700 1 -- 172.21.15.177:0/3817933500 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d40ac5d0 con 0x7fa9c00051c0 2024-09-15T03:36:29.646 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.639+0000 7fa9cbfff700 1 -- 172.21.15.177:0/3817933500 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c000c050 con 0x7fa9c0004810 2024-09-15T03:36:29.646 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.639+0000 7fa9cbfff700 1 -- 172.21.15.177:0/3817933500 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c000c210 con 0x7fa9c0003a60 2024-09-15T03:36:29.646 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.639+0000 7fa9d8c9f700 1 --2- 172.21.15.177:0/3817933500 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa9c0004810 0x7fa9c0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:29.646 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.639+0000 7fa9d94a0700 1 --2- 172.21.15.177:0/3817933500 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fa9c0003a60 0x7fa9c0003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:29.646 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.639+0000 7fa9d9ca1700 1 --2- 172.21.15.177:0/3817933500 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa9c00051c0 0x7fa9c000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:29.646 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.639+0000 7fa9d8c9f700 1 -- 172.21.15.177:0/3817933500 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fa9c0003a60 msgr2=0x7fa9c0003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:36:29.646 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.639+0000 7fa9d8c9f700 1 --2- 172.21.15.177:0/3817933500 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fa9c0003a60 0x7fa9c0003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:29.647 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.639+0000 7fa9d8c9f700 1 -- 172.21.15.177:0/3817933500 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa9c00051c0 msgr2=0x7fa9c000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:36:29.647 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.639+0000 7fa9d8c9f700 1 --2- 172.21.15.177:0/3817933500 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa9c00051c0 0x7fa9c000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:29.647 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.639+0000 7fa9d9ca1700 1 -- 172.21.15.177:0/3817933500 reap_dead start 2024-09-15T03:36:29.647 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.639+0000 7fa9d8c9f700 1 -- 172.21.15.177:0/3817933500 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9c0003570 con 0x7fa9c0004810 2024-09-15T03:36:29.647 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.639+0000 7fa9d8c9f700 1 --2- 172.21.15.177:0/3817933500 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa9c0004810 0x7fa9c0004c80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa9c400c2a0 tx=0x7fa9c4001df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce8b0060d5632364 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:29.647 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.639+0000 7fa9cbfff700 1 -- 172.21.15.177:0/3817933500 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c400cb80 con 0x7fa9c0004810 2024-09-15T03:36:29.647 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.639+0000 7fa9cbfff700 1 -- 172.21.15.177:0/3817933500 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9c400cce0 con 0x7fa9c0004810 2024-09-15T03:36:29.647 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.639+0000 7fa9cbfff700 1 -- 172.21.15.177:0/3817933500 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c4005b70 con 0x7fa9c0004810 2024-09-15T03:36:29.647 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.639+0000 7fa9da4a2700 1 -- 172.21.15.177:0/3817933500 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa9c0004810 msgr2=0x7fa9c0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:29.647 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.639+0000 7fa9da4a2700 1 --2- 172.21.15.177:0/3817933500 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa9c0004810 0x7fa9c0004c80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa9c400c2a0 tx=0x7fa9c4001df0 comp rx=0 tx=0).stop 2024-09-15T03:36:29.647 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.639+0000 7fa9da4a2700 1 -- 172.21.15.177:0/3817933500 shutdown_connections 2024-09-15T03:36:29.648 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.639+0000 7fa9da4a2700 1 --2- 172.21.15.177:0/3817933500 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa9c0004810 0x7fa9c0004c80 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:29.648 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.639+0000 7fa9da4a2700 1 -- 172.21.15.177:0/3817933500 >> 172.21.15.177:0/3817933500 conn(0x7fa9d40b0f60 msgr2=0x7fa9d40b1360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:29.648 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.639+0000 7fa9da4a2700 1 -- 172.21.15.177:0/3817933500 shutdown_connections 2024-09-15T03:36:29.648 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.639+0000 7fa9da4a2700 1 -- 172.21.15.177:0/3817933500 wait complete. 2024-09-15T03:36:29.648 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.643+0000 7fa9da4a2700 1 Processor -- start 2024-09-15T03:36:29.648 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.643+0000 7fa9da4a2700 1 -- start start 2024-09-15T03:36:29.651 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.643+0000 7fa9da4a2700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa9c0004810 0x7fa9d414d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:29.651 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.643+0000 7fa9da4a2700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fa9d414db70 0x7fa9d4138e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:29.651 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.643+0000 7fa9da4a2700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa9d41399e0 0x7fa9d4139e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:29.651 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.643+0000 7fa9da4a2700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d41382d0 con 0x7fa9d41399e0 2024-09-15T03:36:29.651 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.643+0000 7fa9da4a2700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d41394c0 con 0x7fa9c0004810 2024-09-15T03:36:29.651 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.643+0000 7fa9da4a2700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d413a380 con 0x7fa9d414db70 2024-09-15T03:36:29.651 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.643+0000 7fa9d94a0700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa9c0004810 0x7fa9d414d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:29.651 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.643+0000 7fa9d94a0700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa9c0004810 0x7fa9d414d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:54130/0 (socket says 172.21.15.177:54130) 2024-09-15T03:36:29.651 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.643+0000 7fa9d94a0700 1 -- 172.21.15.177:0/734156625 learned_addr learned my addr 172.21.15.177:0/734156625 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-15T03:36:29.651 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.643+0000 7fa9d9ca1700 1 --2- 172.21.15.177:0/734156625 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa9d41399e0 0x7fa9d4139e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:29.652 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.643+0000 7fa9d8c9f700 1 --2- 172.21.15.177:0/734156625 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fa9d414db70 0x7fa9d4138e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:29.652 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.643+0000 7fa9d94a0700 1 -- 172.21.15.177:0/734156625 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fa9d414db70 msgr2=0x7fa9d4138e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:29.652 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.643+0000 7fa9d94a0700 1 --2- 172.21.15.177:0/734156625 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fa9d414db70 0x7fa9d4138e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:29.652 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.643+0000 7fa9d94a0700 1 -- 172.21.15.177:0/734156625 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa9d41399e0 msgr2=0x7fa9d4139e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:29.652 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.643+0000 7fa9d94a0700 1 --2- 172.21.15.177:0/734156625 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa9d41399e0 0x7fa9d4139e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:29.652 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.643+0000 7fa9d94a0700 1 -- 172.21.15.177:0/734156625 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d413beb0 con 0x7fa9c0004810 2024-09-15T03:36:29.652 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.643+0000 7fa9d94a0700 1 --2- 172.21.15.177:0/734156625 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa9c0004810 0x7fa9d414d0f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0011d40 tx=0x7fa9d00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:29.652 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.643+0000 7fa9c97fa700 1 -- 172.21.15.177:0/734156625 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d000fa10 con 0x7fa9c0004810 2024-09-15T03:36:29.652 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.643+0000 7fa9c97fa700 1 -- 172.21.15.177:0/734156625 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9d000fb70 con 0x7fa9c0004810 2024-09-15T03:36:29.652 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.643+0000 7fa9da4a2700 1 -- 172.21.15.177:0/734156625 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9d40aa8f0 con 0x7fa9c0004810 2024-09-15T03:36:29.653 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.643+0000 7fa9c97fa700 1 -- 172.21.15.177:0/734156625 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d0014440 con 0x7fa9c0004810 2024-09-15T03:36:29.653 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.643+0000 7fa9da4a2700 1 -- 172.21.15.177:0/734156625 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9d40aade0 con 0x7fa9c0004810 2024-09-15T03:36:29.653 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.643+0000 7fa9c97fa700 1 -- 172.21.15.177:0/734156625 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fa9d0014870 con 0x7fa9c0004810 2024-09-15T03:36:29.653 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.643+0000 7fa9c97fa700 1 -- 172.21.15.177:0/734156625 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7fa9d0018030 con 0x7fa9c0004810 2024-09-15T03:36:29.653 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.647+0000 7fa9da4a2700 1 -- 172.21.15.177:0/734156625 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9d4139680 con 0x7fa9c0004810 2024-09-15T03:36:29.653 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.651+0000 7fa9c97fa700 1 -- 172.21.15.177:0/734156625 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7fa9d003c7a0 con 0x7fa9c0004810 2024-09-15T03:36:29.721 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-15T03:36:29.715+0000 7fa4cdfbb000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-15T03:36:29.728 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.727+0000 7fa9da4a2700 1 -- 172.21.15.177:0/734156625 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd new", "uuid": "7a87be3c-b6ff-43a4-8672-39d3006a0e2b", "id": 4} v 0) v1 -- 0x7fa9d40716d0 con 0x7fa9c0004810 2024-09-15T03:36:29.738 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.735+0000 7fa9c97fa700 1 -- 172.21.15.177:0/734156625 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7a87be3c-b6ff-43a4-8672-39d3006a0e2b", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa9d002b070 con 0x7fa9c0004810 2024-09-15T03:36:29.738 INFO:teuthology.orchestra.run.smithi177.stdout:4 2024-09-15T03:36:29.740 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.739+0000 7fa9da4a2700 1 -- 172.21.15.177:0/734156625 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa9c0004810 msgr2=0x7fa9d414d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:29.740 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.739+0000 7fa9da4a2700 1 --2- 172.21.15.177:0/734156625 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa9c0004810 0x7fa9d414d0f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0011d40 tx=0x7fa9d00091a0 comp rx=0 tx=0).stop 2024-09-15T03:36:29.740 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.739+0000 7fa9da4a2700 1 -- 172.21.15.177:0/734156625 shutdown_connections 2024-09-15T03:36:29.740 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.739+0000 7fa9da4a2700 1 --2- 172.21.15.177:0/734156625 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa9d41399e0 0x7fa9d4139e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:29.740 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.739+0000 7fa9da4a2700 1 --2- 172.21.15.177:0/734156625 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fa9d414db70 0x7fa9d4138e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:29.740 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.739+0000 7fa9da4a2700 1 --2- 172.21.15.177:0/734156625 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa9c0004810 0x7fa9d414d0f0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:29.740 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.739+0000 7fa9da4a2700 1 -- 172.21.15.177:0/734156625 >> 172.21.15.177:0/734156625 conn(0x7fa9d40b0f60 msgr2=0x7fa9d4133c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:29.740 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.739+0000 7fa9da4a2700 1 -- 172.21.15.177:0/734156625 shutdown_connections 2024-09-15T03:36:29.740 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.739+0000 7fa9da4a2700 1 -- 172.21.15.177:0/734156625 wait complete. 2024-09-15T03:36:29.751 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd new a8f3a63d-45af-41fe-be33-2339d7b9ed96 5 2024-09-15T03:36:29.770 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-15T03:36:29.767+0000 7fa4cdfbb000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-15T03:36:29.861 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.859+0000 7f7ea19d1700 1 Processor -- start 2024-09-15T03:36:29.862 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.859+0000 7f7ea19d1700 1 -- start start 2024-09-15T03:36:29.864 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.859+0000 7f7ea19d1700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f7e9c0b1400 0x7f7e9c0aa860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:29.864 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.859+0000 7f7ea19d1700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f7e9c0b17d0 0x7f7e9c140c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:29.864 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.859+0000 7f7ea19d1700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f7e9c06ecb0 0x7f7e9c06f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:29.864 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.859+0000 7f7ea19d1700 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e9c13b140 con 0x7f7e9c06ecb0 2024-09-15T03:36:29.865 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.859+0000 7f7ea19d1700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e9c0aa440 con 0x7f7e9c0b17d0 2024-09-15T03:36:29.865 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.859+0000 7f7ea19d1700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c0a9570 con 0x7f7e9c0b1400 2024-09-15T03:36:29.865 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.859+0000 7f7e9bfff700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f7e9c0b17d0 0x7f7e9c140c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:29.865 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.859+0000 7f7ea11d0700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f7e9c06ecb0 0x7f7e9c06f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:29.865 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.859+0000 7f7ea09cf700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f7e9c0b1400 0x7f7e9c0aa860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:29.865 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.859+0000 7f7e9bfff700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f7e9c0b17d0 0x7f7e9c140c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:54140/0 (socket says 172.21.15.177:54140) 2024-09-15T03:36:29.865 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.859+0000 7f7e9bfff700 1 -- 172.21.15.177:0/709774493 learned_addr learned my addr 172.21.15.177:0/709774493 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-15T03:36:29.865 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.859+0000 7f7e9bfff700 1 -- 172.21.15.177:0/709774493 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f7e9c0b1400 msgr2=0x7f7e9c0aa860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:36:29.865 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.859+0000 7f7e9bfff700 1 --2- 172.21.15.177:0/709774493 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f7e9c0b1400 0x7f7e9c0aa860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:29.865 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.859+0000 7f7e9bfff700 1 -- 172.21.15.177:0/709774493 >> v2:172.21.15.110:3300/0 conn(0x7f7e9c06ecb0 msgr2=0x7f7e9c06f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:36:29.866 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.859+0000 7f7e9bfff700 1 --2- 172.21.15.177:0/709774493 >> v2:172.21.15.110:3300/0 conn(0x7f7e9c06ecb0 0x7f7e9c06f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:29.866 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.859+0000 7f7e9bfff700 1 -- 172.21.15.177:0/709774493 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e9c0a9770 con 0x7f7e9c0b17d0 2024-09-15T03:36:29.866 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.859+0000 7f7ea11d0700 1 --2- 172.21.15.177:0/709774493 >> v2:172.21.15.110:3300/0 conn(0x7f7e9c06ecb0 0x7f7e9c06f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T03:36:29.866 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.863+0000 7f7e9bfff700 1 --2- 172.21.15.177:0/709774493 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7e9c0b17d0 0x7f7e9c140c60 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c069790 tx=0x7f7e90009220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ab365bf84ecc4e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:29.866 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.863+0000 7f7e9b7fe700 1 -- 172.21.15.177:0/709774493 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e90005c10 con 0x7f7e9c0b17d0 2024-09-15T03:36:29.866 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.863+0000 7f7e9b7fe700 1 -- 172.21.15.177:0/709774493 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e90005d70 con 0x7f7e9c0b17d0 2024-09-15T03:36:29.866 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.863+0000 7f7e9b7fe700 1 -- 172.21.15.177:0/709774493 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e90006260 con 0x7f7e9c0b17d0 2024-09-15T03:36:29.866 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.863+0000 7f7ea19d1700 1 -- 172.21.15.177:0/709774493 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7e9c0b17d0 msgr2=0x7f7e9c140c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:29.866 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.863+0000 7f7ea19d1700 1 --2- 172.21.15.177:0/709774493 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7e9c0b17d0 0x7f7e9c140c60 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c069790 tx=0x7f7e90009220 comp rx=0 tx=0).stop 2024-09-15T03:36:29.866 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.863+0000 7f7ea19d1700 1 -- 172.21.15.177:0/709774493 shutdown_connections 2024-09-15T03:36:29.866 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.863+0000 7f7ea19d1700 1 --2- 172.21.15.177:0/709774493 >> v2:172.21.15.110:3300/0 conn(0x7f7e9c06ecb0 0x7f7e9c06f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:29.866 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.863+0000 7f7ea19d1700 1 --2- 172.21.15.177:0/709774493 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7e9c0b17d0 0x7f7e9c140c60 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:29.867 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.863+0000 7f7ea19d1700 1 --2- 172.21.15.177:0/709774493 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f7e9c0b1400 0x7f7e9c0aa860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:29.867 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.863+0000 7f7ea19d1700 1 -- 172.21.15.177:0/709774493 >> 172.21.15.177:0/709774493 conn(0x7f7e9c0ae270 msgr2=0x7f7e9c0b0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:29.867 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.863+0000 7f7ea19d1700 1 -- 172.21.15.177:0/709774493 shutdown_connections 2024-09-15T03:36:29.867 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.863+0000 7f7ea19d1700 1 -- 172.21.15.177:0/709774493 wait complete. 2024-09-15T03:36:29.869 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.863+0000 7f7ea19d1700 1 Processor -- start 2024-09-15T03:36:29.869 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.863+0000 7f7ea19d1700 1 -- start start 2024-09-15T03:36:29.869 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.863+0000 7f7ea19d1700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7e9c06ecb0 0x7f7e9c1401a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:29.869 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.863+0000 7f7ea19d1700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f7e9c0b1400 0x7f7e9c14d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:29.869 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.863+0000 7f7ea19d1700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7e9c0b17d0 0x7f7e9c14deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:29.869 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.863+0000 7f7ea19d1700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c119840 con 0x7f7e9c0b17d0 2024-09-15T03:36:29.869 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.863+0000 7f7ea19d1700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c13b4d0 con 0x7f7e9c06ecb0 2024-09-15T03:36:29.869 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.863+0000 7f7ea19d1700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c13d1b0 con 0x7f7e9c0b1400 2024-09-15T03:36:29.869 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.863+0000 7f7ea09cf700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7e9c06ecb0 0x7f7e9c1401a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:29.870 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.867+0000 7f7ea09cf700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7e9c06ecb0 0x7f7e9c1401a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:54150/0 (socket says 172.21.15.177:54150) 2024-09-15T03:36:29.870 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.867+0000 7f7ea11d0700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7e9c0b17d0 0x7f7e9c14deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:29.870 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.867+0000 7f7ea09cf700 1 -- 172.21.15.177:0/386212271 learned_addr learned my addr 172.21.15.177:0/386212271 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-15T03:36:29.870 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.867+0000 7f7e9bfff700 1 --2- 172.21.15.177:0/386212271 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f7e9c0b1400 0x7f7e9c14d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:29.870 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.867+0000 7f7ea09cf700 1 -- 172.21.15.177:0/386212271 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f7e9c0b1400 msgr2=0x7f7e9c14d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:29.870 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.867+0000 7f7ea09cf700 1 --2- 172.21.15.177:0/386212271 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f7e9c0b1400 0x7f7e9c14d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:29.871 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.867+0000 7f7ea09cf700 1 -- 172.21.15.177:0/386212271 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7e9c0b17d0 msgr2=0x7f7e9c14deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:29.871 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.867+0000 7f7ea09cf700 1 --2- 172.21.15.177:0/386212271 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7e9c0b17d0 0x7f7e9c14deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:29.872 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.867+0000 7f7ea09cf700 1 -- 172.21.15.177:0/386212271 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e8c009420 con 0x7f7e9c06ecb0 2024-09-15T03:36:29.872 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.867+0000 7f7ea11d0700 1 --2- 172.21.15.177:0/386212271 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7e9c0b17d0 0x7f7e9c14deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T03:36:29.872 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.867+0000 7f7ea09cf700 1 --2- 172.21.15.177:0/386212271 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7e9c06ecb0 0x7f7e9c1401a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7e94009f10 tx=0x7f7e9400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:29.872 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.867+0000 7f7e98ff9700 1 -- 172.21.15.177:0/386212271 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9400cc20 con 0x7f7e9c06ecb0 2024-09-15T03:36:29.872 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.867+0000 7f7e98ff9700 1 -- 172.21.15.177:0/386212271 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e9400f040 con 0x7f7e9c06ecb0 2024-09-15T03:36:29.872 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.867+0000 7f7e98ff9700 1 -- 172.21.15.177:0/386212271 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e94014480 con 0x7f7e9c06ecb0 2024-09-15T03:36:29.872 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.867+0000 7f7ea19d1700 1 -- 172.21.15.177:0/386212271 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e90009480 con 0x7f7e9c06ecb0 2024-09-15T03:36:29.872 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.867+0000 7f7ea19d1700 1 -- 172.21.15.177:0/386212271 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e9c13bcb0 con 0x7f7e9c06ecb0 2024-09-15T03:36:29.872 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.867+0000 7f7e98ff9700 1 -- 172.21.15.177:0/386212271 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f7e94014ac0 con 0x7f7e9c06ecb0 2024-09-15T03:36:29.872 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.867+0000 7f7e98ff9700 1 -- 172.21.15.177:0/386212271 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1296+0+0 (secure 0 0 0) 0x7f7e94013960 con 0x7f7e9c06ecb0 2024-09-15T03:36:29.873 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.867+0000 7f7ea19d1700 1 -- 172.21.15.177:0/386212271 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e9c0ad590 con 0x7f7e9c06ecb0 2024-09-15T03:36:29.873 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.871+0000 7f7e98ff9700 1 -- 172.21.15.177:0/386212271 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f7e9c0ad590 con 0x7f7e9c06ecb0 2024-09-15T03:36:29.952 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-15T03:36:29.947+0000 7fa4cdfbb000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-15T03:36:29.953 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.951+0000 7f7ea19d1700 1 -- 172.21.15.177:0/386212271 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a8f3a63d-45af-41fe-be33-2339d7b9ed96", "id": 5} v 0) v1 -- 0x7f7e9c0716d0 con 0x7f7e9c06ecb0 2024-09-15T03:36:29.962 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.959+0000 7f7e98ff9700 1 -- 172.21.15.177:0/386212271 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a8f3a63d-45af-41fe-be33-2339d7b9ed96", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7e94018020 con 0x7f7e9c06ecb0 2024-09-15T03:36:29.962 INFO:teuthology.orchestra.run.smithi177.stdout:5 2024-09-15T03:36:29.964 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.963+0000 7f7ea19d1700 1 -- 172.21.15.177:0/386212271 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7e9c06ecb0 msgr2=0x7f7e9c1401a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:29.964 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.963+0000 7f7ea19d1700 1 --2- 172.21.15.177:0/386212271 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7e9c06ecb0 0x7f7e9c1401a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7e94009f10 tx=0x7f7e9400b040 comp rx=0 tx=0).stop 2024-09-15T03:36:29.964 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.963+0000 7f7ea19d1700 1 -- 172.21.15.177:0/386212271 shutdown_connections 2024-09-15T03:36:29.964 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.963+0000 7f7ea19d1700 1 --2- 172.21.15.177:0/386212271 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7e9c0b17d0 0x7f7e9c14deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:29.964 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.963+0000 7f7ea19d1700 1 --2- 172.21.15.177:0/386212271 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f7e9c0b1400 0x7f7e9c14d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:29.964 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.963+0000 7f7ea19d1700 1 --2- 172.21.15.177:0/386212271 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7e9c06ecb0 0x7f7e9c1401a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:29.964 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.963+0000 7f7ea19d1700 1 -- 172.21.15.177:0/386212271 >> 172.21.15.177:0/386212271 conn(0x7f7e9c0ae270 msgr2=0x7f7e9c068ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:29.964 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.963+0000 7f7ea19d1700 1 -- 172.21.15.177:0/386212271 shutdown_connections 2024-09-15T03:36:29.964 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:29.963+0000 7f7ea19d1700 1 -- 172.21.15.177:0/386212271 wait complete. 2024-09-15T03:36:29.976 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd new a61b52dd-273d-47b4-bf92-0610cce86e1e 6 2024-09-15T03:36:30.011 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-15T03:36:30.007+0000 7fa4cdfbb000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-15T03:36:30.072 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-15T03:36:30.067+0000 7fa4cdfbb000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-15T03:36:30.080 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.079+0000 7f017279d700 1 Processor -- start 2024-09-15T03:36:30.080 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.079+0000 7f017279d700 1 -- start start 2024-09-15T03:36:30.082 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.079+0000 7f017279d700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f016c06f420 0x7f016c13d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:30.082 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.079+0000 7f017279d700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f016c06d940 0x7f016c0abec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:30.082 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.079+0000 7f017279d700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f016c06d4b0 con 0x7f016c13dde0 2024-09-15T03:36:30.082 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.079+0000 7f017279d700 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f016c06f7f0 con 0x7f016c06d940 2024-09-15T03:36:30.082 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.079+0000 7f017279d700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f016c0ad640 con 0x7f016c06f420 2024-09-15T03:36:30.082 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.079+0000 7f0170f9a700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f016c13dde0 0x7f016c0ab7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:37058/0 (socket says 172.21.15.177:37058) 2024-09-15T03:36:30.082 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.079+0000 7f0170f9a700 1 -- 172.21.15.177:0/3167569846 learned_addr learned my addr 172.21.15.177:0/3167569846 (peer_addr_for_me v1:172.21.15.177:0/0) 2024-09-15T03:36:30.083 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.079+0000 7f017179b700 1 --2- 172.21.15.177:0/3167569846 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f016c06f420 0x7f016c13d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:30.083 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.079+0000 7f0171f9c700 1 --2- 172.21.15.177:0/3167569846 >> v2:172.21.15.110:3300/0 conn(0x7f016c06d940 0x7f016c0abec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:30.083 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.079+0000 7f0163fff700 1 -- 172.21.15.177:0/3167569846 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1283472531 0 0) 0x7f016c06d4b0 con 0x7f016c13dde0 2024-09-15T03:36:30.083 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.079+0000 7f0163fff700 1 -- 172.21.15.177:0/3167569846 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0158003570 con 0x7f016c13dde0 2024-09-15T03:36:30.083 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.079+0000 7f017179b700 1 -- 172.21.15.177:0/3167569846 >> v2:172.21.15.110:3300/0 conn(0x7f016c06d940 msgr2=0x7f016c0abec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:36:30.083 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.079+0000 7f017179b700 1 --2- 172.21.15.177:0/3167569846 >> v2:172.21.15.110:3300/0 conn(0x7f016c06d940 0x7f016c0abec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:30.083 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.079+0000 7f017179b700 1 -- 172.21.15.177:0/3167569846 >> v1:172.21.15.177:6789/0 conn(0x7f016c13dde0 legacy=0x7f016c0ab7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:30.083 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.079+0000 7f0163fff700 1 -- 172.21.15.177:0/3167569846 <== mon.1 v1:172.21.15.177:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 146370803 0 0) 0x7f0158003570 con 0x7f016c13dde0 2024-09-15T03:36:30.083 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.079+0000 7f017179b700 1 -- 172.21.15.177:0/3167569846 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f016c06e9f0 con 0x7f016c06f420 2024-09-15T03:36:30.084 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.079+0000 7f017179b700 1 --2- 172.21.15.177:0/3167569846 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f016c06f420 0x7f016c13d8a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f01680098e0 tx=0x7f01680091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=776356a958118c34 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:30.084 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.083+0000 7f0163fff700 1 -- 172.21.15.177:0/3167569846 <== mon.2 v2:172.21.15.110:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f015c001470 con 0x7f016c06f420 2024-09-15T03:36:30.085 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.083+0000 7f0163fff700 1 -- 172.21.15.177:0/3167569846 <== mon.2 v2:172.21.15.110:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0168011420 con 0x7f016c06f420 2024-09-15T03:36:30.085 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.083+0000 7f0163fff700 1 -- 172.21.15.177:0/3167569846 <== mon.2 v2:172.21.15.110:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01680117a0 con 0x7f016c06f420 2024-09-15T03:36:30.085 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.083+0000 7f017279d700 1 -- 172.21.15.177:0/3167569846 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f016c06f420 msgr2=0x7f016c13d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:30.085 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.083+0000 7f017279d700 1 --2- 172.21.15.177:0/3167569846 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f016c06f420 0x7f016c13d8a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f01680098e0 tx=0x7f01680091a0 comp rx=0 tx=0).stop 2024-09-15T03:36:30.085 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.083+0000 7f017279d700 1 -- 172.21.15.177:0/3167569846 shutdown_connections 2024-09-15T03:36:30.085 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.083+0000 7f017279d700 1 --2- 172.21.15.177:0/3167569846 >> v2:172.21.15.110:3300/0 conn(0x7f016c06d940 0x7f016c0abec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:30.085 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.083+0000 7f017279d700 1 --2- 172.21.15.177:0/3167569846 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f016c06f420 0x7f016c13d8a0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:30.085 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.083+0000 7f017279d700 1 -- 172.21.15.177:0/3167569846 >> 172.21.15.177:0/3167569846 conn(0x7f016c0b0f60 msgr2=0x7f016c0b1360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:30.085 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.083+0000 7f017279d700 1 -- 172.21.15.177:0/3167569846 shutdown_connections 2024-09-15T03:36:30.085 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.083+0000 7f017279d700 1 -- 172.21.15.177:0/3167569846 wait complete. 2024-09-15T03:36:30.086 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.083+0000 7f017279d700 1 Processor -- start 2024-09-15T03:36:30.086 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.083+0000 7f017279d700 1 -- start start 2024-09-15T03:36:30.088 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.083+0000 7f017279d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f016c06d940 0x7f016c14d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:30.088 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.083+0000 7f017279d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f016c06f420 0x7f016c14dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:30.088 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.083+0000 7f017279d700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f016c13dde0 0x7f016c133b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:30.088 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.083+0000 7f017279d700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f016c13b0e0 con 0x7f016c06d940 2024-09-15T03:36:30.088 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.083+0000 7f017279d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f016c0a6170 con 0x7f016c06f420 2024-09-15T03:36:30.088 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.083+0000 7f017279d700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f016c13a300 con 0x7f016c13dde0 2024-09-15T03:36:30.088 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.087+0000 7f0170f9a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f016c06f420 0x7f016c14dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:30.088 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.087+0000 7f017179b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f016c06d940 0x7f016c14d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:30.088 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.087+0000 7f0170f9a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f016c06f420 0x7f016c14dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:54158/0 (socket says 172.21.15.177:54158) 2024-09-15T03:36:30.088 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.087+0000 7f0170f9a700 1 -- 172.21.15.177:0/3855346931 learned_addr learned my addr 172.21.15.177:0/3855346931 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-15T03:36:30.088 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.087+0000 7f0171f9c700 1 --2- 172.21.15.177:0/3855346931 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f016c13dde0 0x7f016c133b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:30.088 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.087+0000 7f017179b700 1 -- 172.21.15.177:0/3855346931 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f016c13dde0 msgr2=0x7f016c133b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:30.089 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.087+0000 7f017179b700 1 --2- 172.21.15.177:0/3855346931 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f016c13dde0 0x7f016c133b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:30.089 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.087+0000 7f017179b700 1 -- 172.21.15.177:0/3855346931 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f016c06f420 msgr2=0x7f016c14dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:30.089 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.087+0000 7f017179b700 1 --2- 172.21.15.177:0/3855346931 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f016c06f420 0x7f016c14dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:30.089 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.087+0000 7f017179b700 1 -- 172.21.15.177:0/3855346931 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0168009420 con 0x7f016c06d940 2024-09-15T03:36:30.089 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.087+0000 7f0170f9a700 1 --2- 172.21.15.177:0/3855346931 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f016c06f420 0x7f016c14dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T03:36:30.089 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.087+0000 7f017179b700 1 --2- 172.21.15.177:0/3855346931 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f016c06d940 0x7f016c14d730 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0168015000 tx=0x7f0168016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:30.090 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.087+0000 7f01617fa700 1 -- 172.21.15.177:0/3855346931 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f016801caf0 con 0x7f016c06d940 2024-09-15T03:36:30.090 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.087+0000 7f01617fa700 1 -- 172.21.15.177:0/3855346931 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f016801cc50 con 0x7f016c06d940 2024-09-15T03:36:30.090 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.087+0000 7f017279d700 1 -- 172.21.15.177:0/3855346931 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f016c0a44d0 con 0x7f016c06d940 2024-09-15T03:36:30.090 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.087+0000 7f01617fa700 1 -- 172.21.15.177:0/3855346931 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f016801a450 con 0x7f016c06d940 2024-09-15T03:36:30.090 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.087+0000 7f017279d700 1 -- 172.21.15.177:0/3855346931 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f016c1340a0 con 0x7f016c06d940 2024-09-15T03:36:30.090 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.087+0000 7f01617fa700 1 -- 172.21.15.177:0/3855346931 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f016801a820 con 0x7f016c06d940 2024-09-15T03:36:30.090 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.087+0000 7f01617fa700 1 -- 172.21.15.177:0/3855346931 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1411+0+0 (secure 0 0 0) 0x7f016801aad0 con 0x7f016c06d940 2024-09-15T03:36:30.091 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.087+0000 7f017279d700 1 -- 172.21.15.177:0/3855346931 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f016c13aeb0 con 0x7f016c06d940 2024-09-15T03:36:30.095 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.095+0000 7f01617fa700 1 -- 172.21.15.177:0/3855346931 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f016c13aeb0 con 0x7f016c06d940 2024-09-15T03:36:30.174 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.171+0000 7f017279d700 1 -- 172.21.15.177:0/3855346931 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a61b52dd-273d-47b4-bf92-0610cce86e1e", "id": 6} v 0) v1 -- 0x7f016c0716d0 con 0x7f016c06d940 2024-09-15T03:36:30.179 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.179+0000 7f01617fa700 1 -- 172.21.15.177:0/3855346931 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a61b52dd-273d-47b4-bf92-0610cce86e1e", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0168029070 con 0x7f016c06d940 2024-09-15T03:36:30.180 INFO:teuthology.orchestra.run.smithi177.stdout:6 2024-09-15T03:36:30.181 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.179+0000 7f017279d700 1 -- 172.21.15.177:0/3855346931 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f016c06d940 msgr2=0x7f016c14d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:30.181 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.179+0000 7f017279d700 1 --2- 172.21.15.177:0/3855346931 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f016c06d940 0x7f016c14d730 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0168015000 tx=0x7f0168016000 comp rx=0 tx=0).stop 2024-09-15T03:36:30.181 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.179+0000 7f017279d700 1 -- 172.21.15.177:0/3855346931 shutdown_connections 2024-09-15T03:36:30.181 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.179+0000 7f017279d700 1 --2- 172.21.15.177:0/3855346931 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f016c13dde0 0x7f016c133b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:30.182 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.179+0000 7f017279d700 1 --2- 172.21.15.177:0/3855346931 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f016c06f420 0x7f016c14dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:30.182 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.179+0000 7f017279d700 1 --2- 172.21.15.177:0/3855346931 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f016c06d940 0x7f016c14d730 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:30.182 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.179+0000 7f017279d700 1 -- 172.21.15.177:0/3855346931 >> 172.21.15.177:0/3855346931 conn(0x7f016c0b0f60 msgr2=0x7f016c0afab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:30.182 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.179+0000 7f017279d700 1 -- 172.21.15.177:0/3855346931 shutdown_connections 2024-09-15T03:36:30.182 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.179+0000 7f017279d700 1 -- 172.21.15.177:0/3855346931 wait complete. 2024-09-15T03:36:30.190 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-15T03:36:30.187+0000 7febd9357000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-15T03:36:30.194 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd new 1a7d5a9f-6b41-47e1-b0ef-45f4d638fd65 7 2024-09-15T03:36:30.248 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-15T03:36:30.247+0000 7febd9357000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-15T03:36:30.297 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.295+0000 7fd38300a700 1 Processor -- start 2024-09-15T03:36:30.297 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.295+0000 7fd38300a700 1 -- start start 2024-09-15T03:36:30.297 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.295+0000 7fd38300a700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fd37c0b1450 0x7fd37c140cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:30.297 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.295+0000 7fd38300a700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fd37c0af200 0x7fd37c148280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:30.297 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.295+0000 7fd38300a700 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd37c0a5370 con 0x7fd37c141210 2024-09-15T03:36:30.297 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.295+0000 7fd38300a700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd37c141690 con 0x7fd37c0b1450 2024-09-15T03:36:30.298 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.295+0000 7fd38300a700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd37c0b1820 con 0x7fd37c0af200 2024-09-15T03:36:30.298 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.295+0000 7fd380da6700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fd37c0b1450 0x7fd37c140cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:30.298 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.295+0000 7fd380da6700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fd37c0b1450 0x7fd37c140cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:54160/0 (socket says 172.21.15.177:54160) 2024-09-15T03:36:30.298 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.295+0000 7fd380da6700 1 -- 172.21.15.177:0/1039108655 learned_addr learned my addr 172.21.15.177:0/1039108655 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-15T03:36:30.298 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.295+0000 7fd3815a7700 1 --2- 172.21.15.177:0/1039108655 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fd37c0af200 0x7fd37c148280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:30.299 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.295+0000 7fd380da6700 1 -- 172.21.15.177:0/1039108655 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fd37c0af200 msgr2=0x7fd37c148280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:36:30.299 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.295+0000 7fd37b7fe700 1 -- 172.21.15.177:0/1039108655 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3835869896 0 0) 0x7fd37c0a5370 con 0x7fd37c141210 2024-09-15T03:36:30.299 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.295+0000 7fd380da6700 1 --2- 172.21.15.177:0/1039108655 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fd37c0af200 0x7fd37c148280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:30.300 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.295+0000 7fd380da6700 1 -- 172.21.15.177:0/1039108655 >> v1:172.21.15.110:6789/0 conn(0x7fd37c141210 legacy=0x7fd37c0aeaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:30.300 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.295+0000 7fd380da6700 1 -- 172.21.15.177:0/1039108655 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd37c0a6760 con 0x7fd37c0b1450 2024-09-15T03:36:30.300 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.295+0000 7fd380da6700 1 --2- 172.21.15.177:0/1039108655 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd37c0b1450 0x7fd37c140cd0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd36c0099b0 tx=0x7fd36c009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f87fa72da776ae55 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:30.300 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.295+0000 7fd37b7fe700 1 -- 172.21.15.177:0/1039108655 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36c009ea0 con 0x7fd37c0b1450 2024-09-15T03:36:30.300 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.295+0000 7fd37b7fe700 1 -- 172.21.15.177:0/1039108655 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd36c00b860 con 0x7fd37c0b1450 2024-09-15T03:36:30.300 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.295+0000 7fd37b7fe700 1 -- 172.21.15.177:0/1039108655 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36c00bc00 con 0x7fd37c0b1450 2024-09-15T03:36:30.300 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.299+0000 7fd38300a700 1 -- 172.21.15.177:0/1039108655 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd37c0b1450 msgr2=0x7fd37c140cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:30.300 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.299+0000 7fd38300a700 1 --2- 172.21.15.177:0/1039108655 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd37c0b1450 0x7fd37c140cd0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd36c0099b0 tx=0x7fd36c009250 comp rx=0 tx=0).stop 2024-09-15T03:36:30.300 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.299+0000 7fd38300a700 1 -- 172.21.15.177:0/1039108655 shutdown_connections 2024-09-15T03:36:30.300 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.299+0000 7fd38300a700 1 --2- 172.21.15.177:0/1039108655 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fd37c0af200 0x7fd37c148280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:30.300 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.299+0000 7fd38300a700 1 --2- 172.21.15.177:0/1039108655 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd37c0b1450 0x7fd37c140cd0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:30.300 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.299+0000 7fd38300a700 1 -- 172.21.15.177:0/1039108655 >> 172.21.15.177:0/1039108655 conn(0x7fd37c06f140 msgr2=0x7fd37c06f540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:30.301 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.299+0000 7fd38300a700 1 -- 172.21.15.177:0/1039108655 shutdown_connections 2024-09-15T03:36:30.301 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.299+0000 7fd38300a700 1 -- 172.21.15.177:0/1039108655 wait complete. 2024-09-15T03:36:30.303 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.299+0000 7fd38300a700 1 Processor -- start 2024-09-15T03:36:30.303 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.299+0000 7fd38300a700 1 -- start start 2024-09-15T03:36:30.303 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.299+0000 7fd38300a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd37c0af200 0x7fd37c0a7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:30.303 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.299+0000 7fd38300a700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fd37c0b1450 0x7fd37c0a7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:30.303 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.299+0000 7fd38300a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd37c141210 0x7fd37c151070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:30.303 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.299+0000 7fd38300a700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd37c150be0 con 0x7fd37c0af200 2024-09-15T03:36:30.303 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.299+0000 7fd38300a700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd37c0a9610 con 0x7fd37c141210 2024-09-15T03:36:30.303 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.299+0000 7fd38300a700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd37c0b29a0 con 0x7fd37c0b1450 2024-09-15T03:36:30.303 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.299+0000 7fd3815a7700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd37c141210 0x7fd37c151070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:30.304 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.299+0000 7fd3815a7700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd37c141210 0x7fd37c151070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:54164/0 (socket says 172.21.15.177:54164) 2024-09-15T03:36:30.304 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.299+0000 7fd3815a7700 1 -- 172.21.15.177:0/3223092504 learned_addr learned my addr 172.21.15.177:0/3223092504 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-15T03:36:30.304 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.299+0000 7fd380da6700 1 --2- 172.21.15.177:0/3223092504 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd37c0af200 0x7fd37c0a7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:30.304 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.299+0000 7fd37bfff700 1 --2- 172.21.15.177:0/3223092504 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fd37c0b1450 0x7fd37c0a7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:30.304 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.299+0000 7fd3815a7700 1 -- 172.21.15.177:0/3223092504 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fd37c0b1450 msgr2=0x7fd37c0a7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:30.304 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.299+0000 7fd3815a7700 1 --2- 172.21.15.177:0/3223092504 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fd37c0b1450 0x7fd37c0a7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:30.304 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.299+0000 7fd3815a7700 1 -- 172.21.15.177:0/3223092504 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd37c0af200 msgr2=0x7fd37c0a7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:30.304 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.299+0000 7fd3815a7700 1 --2- 172.21.15.177:0/3223092504 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd37c0af200 0x7fd37c0a7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:30.304 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.299+0000 7fd3815a7700 1 -- 172.21.15.177:0/3223092504 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd36c0094f0 con 0x7fd37c141210 2024-09-15T03:36:30.304 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.299+0000 7fd3815a7700 1 --2- 172.21.15.177:0/3223092504 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd37c141210 0x7fd37c151070 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd374009e90 tx=0x7fd37400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:30.304 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.299+0000 7fd378ff9700 1 -- 172.21.15.177:0/3223092504 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37400cb70 con 0x7fd37c141210 2024-09-15T03:36:30.305 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.299+0000 7fd378ff9700 1 -- 172.21.15.177:0/3223092504 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd37400f040 con 0x7fd37c141210 2024-09-15T03:36:30.305 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.299+0000 7fd378ff9700 1 -- 172.21.15.177:0/3223092504 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd374014480 con 0x7fd37c141210 2024-09-15T03:36:30.305 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.299+0000 7fd38300a700 1 -- 172.21.15.177:0/3223092504 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd37c0a76d0 con 0x7fd37c141210 2024-09-15T03:36:30.305 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.299+0000 7fd38300a700 1 -- 172.21.15.177:0/3223092504 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd37c151750 con 0x7fd37c141210 2024-09-15T03:36:30.305 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.299+0000 7fd378ff9700 1 -- 172.21.15.177:0/3223092504 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fd374014ac0 con 0x7fd37c141210 2024-09-15T03:36:30.305 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.299+0000 7fd378ff9700 1 -- 172.21.15.177:0/3223092504 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1526+0+0 (secure 0 0 0) 0x7fd374013a50 con 0x7fd37c141210 2024-09-15T03:36:30.305 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.303+0000 7fd38300a700 1 -- 172.21.15.177:0/3223092504 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd37c150d50 con 0x7fd37c141210 2024-09-15T03:36:30.307 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.307+0000 7fd378ff9700 1 -- 172.21.15.177:0/3223092504 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7fd374018020 con 0x7fd37c141210 2024-09-15T03:36:30.364 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-15T03:36:30.363+0000 7febd9357000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-15T03:36:30.385 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.383+0000 7fd38300a700 1 -- 172.21.15.177:0/3223092504 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd new", "uuid": "1a7d5a9f-6b41-47e1-b0ef-45f4d638fd65", "id": 7} v 0) v1 -- 0x7fd37c0716d0 con 0x7fd37c141210 2024-09-15T03:36:30.391 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.391+0000 7fd378ff9700 1 -- 172.21.15.177:0/3223092504 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1a7d5a9f-6b41-47e1-b0ef-45f4d638fd65", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd37401d400 con 0x7fd37c141210 2024-09-15T03:36:30.391 INFO:teuthology.orchestra.run.smithi177.stdout:7 2024-09-15T03:36:30.393 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.391+0000 7fd38300a700 1 -- 172.21.15.177:0/3223092504 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd37c141210 msgr2=0x7fd37c151070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:30.393 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.391+0000 7fd38300a700 1 --2- 172.21.15.177:0/3223092504 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd37c141210 0x7fd37c151070 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd374009e90 tx=0x7fd37400b040 comp rx=0 tx=0).stop 2024-09-15T03:36:30.393 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.391+0000 7fd38300a700 1 -- 172.21.15.177:0/3223092504 shutdown_connections 2024-09-15T03:36:30.393 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.391+0000 7fd38300a700 1 --2- 172.21.15.177:0/3223092504 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd37c141210 0x7fd37c151070 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:30.393 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.391+0000 7fd38300a700 1 --2- 172.21.15.177:0/3223092504 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fd37c0b1450 0x7fd37c0a7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:30.393 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.391+0000 7fd38300a700 1 --2- 172.21.15.177:0/3223092504 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd37c0af200 0x7fd37c0a7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:30.393 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.391+0000 7fd38300a700 1 -- 172.21.15.177:0/3223092504 >> 172.21.15.177:0/3223092504 conn(0x7fd37c06f140 msgr2=0x7fd37c13cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:30.393 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.391+0000 7fd38300a700 1 -- 172.21.15.177:0/3223092504 shutdown_connections 2024-09-15T03:36:30.394 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-15T03:36:30.391+0000 7fd38300a700 1 -- 172.21.15.177:0/3223092504 wait complete. 2024-09-15T03:36:30.405 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-15T03:36:30.405 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-15T03:36:30.407 INFO:tasks.ceph.osd.0:Started 2024-09-15T03:36:30.407 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-15T03:36:30.407 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-15T03:36:30.409 INFO:tasks.ceph.osd.1:Started 2024-09-15T03:36:30.409 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-15T03:36:30.409 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-15T03:36:30.411 INFO:tasks.ceph.osd.2:Started 2024-09-15T03:36:30.411 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-15T03:36:30.411 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-15T03:36:30.413 INFO:tasks.ceph.osd.3:Started 2024-09-15T03:36:30.413 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-15T03:36:30.413 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-15T03:36:30.415 INFO:tasks.ceph.osd.4:Started 2024-09-15T03:36:30.415 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-15T03:36:30.415 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-15T03:36:30.417 INFO:tasks.ceph.osd.5:Started 2024-09-15T03:36:30.417 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-15T03:36:30.418 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-15T03:36:30.420 INFO:tasks.ceph.osd.6:Started 2024-09-15T03:36:30.420 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-15T03:36:30.420 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-15T03:36:30.422 INFO:tasks.ceph.osd.7:Started 2024-09-15T03:36:30.422 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T03:36:30.480 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-15T03:36:30.475+0000 7fa4cdfbb000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-15T03:36:30.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.523+0000 7f9999b98700 1 Processor -- start 2024-09-15T03:36:30.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.523+0000 7f9999b98700 1 -- start start 2024-09-15T03:36:30.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.523+0000 7f9999b98700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f9994068ce0 0x7f99940a8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:30.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.523+0000 7f9999b98700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f999406d180 0x7f99940a89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:30.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.523+0000 7f9999b98700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f999406db30 0x7f99940ab9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:30.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.523+0000 7f9999b98700 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99940690b0 con 0x7f9994068ce0 2024-09-15T03:36:30.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.523+0000 7f9999b98700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f999406e8f0 con 0x7f999406db30 2024-09-15T03:36:30.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.523+0000 7f9999b98700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f999406ebd0 con 0x7f999406d180 2024-09-15T03:36:30.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.523+0000 7f99937fe700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f9994068ce0 0x7f99940a8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:30.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.523+0000 7f9992ffd700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f999406d180 0x7f99940a89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:30.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.523+0000 7f99937fe700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f9994068ce0 0x7f99940a8490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52956/0 (socket says 172.21.15.110:52956) 2024-09-15T03:36:30.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.523+0000 7f9992ffd700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f999406d180 0x7f99940a89d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3301/0 says I am v2:172.21.15.110:34586/0 (socket says 172.21.15.110:34586) 2024-09-15T03:36:30.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.523+0000 7f99937fe700 1 -- 172.21.15.110:0/3142758460 learned_addr learned my addr 172.21.15.110:0/3142758460 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:36:30.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.523+0000 7f9993fff700 1 --2- 172.21.15.110:0/3142758460 >> v2:172.21.15.177:3300/0 conn(0x7f999406db30 0x7f99940ab9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:30.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.523+0000 7f9992ffd700 1 -- 172.21.15.110:0/3142758460 >> v2:172.21.15.177:3300/0 conn(0x7f999406db30 msgr2=0x7f99940ab9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:36:30.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.523+0000 7f9992ffd700 1 --2- 172.21.15.110:0/3142758460 >> v2:172.21.15.177:3300/0 conn(0x7f999406db30 0x7f99940ab9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:30.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.523+0000 7f9992ffd700 1 -- 172.21.15.110:0/3142758460 >> v2:172.21.15.110:3300/0 conn(0x7f9994068ce0 msgr2=0x7f99940a8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:36:30.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.523+0000 7f9992ffd700 1 --2- 172.21.15.110:0/3142758460 >> v2:172.21.15.110:3300/0 conn(0x7f9994068ce0 0x7f99940a8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:30.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.523+0000 7f9992ffd700 1 -- 172.21.15.110:0/3142758460 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f999406d610 con 0x7f999406d180 2024-09-15T03:36:30.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.523+0000 7f9992ffd700 1 --2- 172.21.15.110:0/3142758460 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f999406d180 0x7f99940a89d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f99940ad1e0 tx=0x7f9984009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edecb6d6e800a556 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:30.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.523+0000 7f99927fc700 1 -- 172.21.15.110:0/3142758460 <== mon.2 v2:172.21.15.110:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f998400e070 con 0x7f999406d180 2024-09-15T03:36:30.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.523+0000 7f99927fc700 1 -- 172.21.15.110:0/3142758460 <== mon.2 v2:172.21.15.110:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f998400bb60 con 0x7f999406d180 2024-09-15T03:36:30.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.523+0000 7f99927fc700 1 -- 172.21.15.110:0/3142758460 <== mon.2 v2:172.21.15.110:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9984012430 con 0x7f999406d180 2024-09-15T03:36:30.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.523+0000 7f9999b98700 1 -- 172.21.15.110:0/3142758460 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f999406d180 msgr2=0x7f99940a89d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:30.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.523+0000 7f9999b98700 1 --2- 172.21.15.110:0/3142758460 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f999406d180 0x7f99940a89d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f99940ad1e0 tx=0x7f9984009780 comp rx=0 tx=0).stop 2024-09-15T03:36:30.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.523+0000 7f9999b98700 1 -- 172.21.15.110:0/3142758460 shutdown_connections 2024-09-15T03:36:30.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.523+0000 7f9999b98700 1 --2- 172.21.15.110:0/3142758460 >> v2:172.21.15.177:3300/0 conn(0x7f999406db30 0x7f99940ab9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:30.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.523+0000 7f9999b98700 1 --2- 172.21.15.110:0/3142758460 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f999406d180 0x7f99940a89d0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:30.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.523+0000 7f9999b98700 1 --2- 172.21.15.110:0/3142758460 >> v2:172.21.15.110:3300/0 conn(0x7f9994068ce0 0x7f99940a8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:30.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.523+0000 7f9999b98700 1 -- 172.21.15.110:0/3142758460 >> 172.21.15.110:0/3142758460 conn(0x7f99940b1150 msgr2=0x7f99940b1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:30.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.523+0000 7f9999b98700 1 -- 172.21.15.110:0/3142758460 shutdown_connections 2024-09-15T03:36:30.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.523+0000 7f9999b98700 1 -- 172.21.15.110:0/3142758460 wait complete. 2024-09-15T03:36:30.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.527+0000 7f9999b98700 1 Processor -- start 2024-09-15T03:36:30.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.527+0000 7f9999b98700 1 -- start start 2024-09-15T03:36:30.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.527+0000 7f9999b98700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9994068ce0 0x7f9994118dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:30.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.527+0000 7f9999b98700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f999406d180 0x7f99940a4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:30.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.527+0000 7f9999b98700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f999406db30 0x7f99940a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:30.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.527+0000 7f9999b98700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99940a4600 con 0x7f9994068ce0 2024-09-15T03:36:30.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.527+0000 7f9999b98700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f999411a070 con 0x7f999406d180 2024-09-15T03:36:30.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.527+0000 7f9999b98700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f999411a350 con 0x7f999406db30 2024-09-15T03:36:30.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.527+0000 7f9993fff700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f999406db30 0x7f99940a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:30.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.527+0000 7f9993fff700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f999406db30 0x7f99940a4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3301/0 says I am v2:172.21.15.110:34592/0 (socket says 172.21.15.110:34592) 2024-09-15T03:36:30.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.527+0000 7f9993fff700 1 -- 172.21.15.110:0/1036484023 learned_addr learned my addr 172.21.15.110:0/1036484023 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:36:30.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.527+0000 7f99937fe700 1 --2- 172.21.15.110:0/1036484023 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9994068ce0 0x7f9994118dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:30.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.527+0000 7f9992ffd700 1 --2- 172.21.15.110:0/1036484023 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f999406d180 0x7f99940a4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:30.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.527+0000 7f9993fff700 1 -- 172.21.15.110:0/1036484023 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f999406d180 msgr2=0x7f99940a4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:30.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.527+0000 7f9993fff700 1 --2- 172.21.15.110:0/1036484023 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f999406d180 0x7f99940a4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:30.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.527+0000 7f9993fff700 1 -- 172.21.15.110:0/1036484023 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9994068ce0 msgr2=0x7f9994118dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:30.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.527+0000 7f9993fff700 1 --2- 172.21.15.110:0/1036484023 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9994068ce0 0x7f9994118dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:30.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.527+0000 7f9993fff700 1 -- 172.21.15.110:0/1036484023 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9984009a20 con 0x7f999406db30 2024-09-15T03:36:30.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.527+0000 7f9993fff700 1 --2- 172.21.15.110:0/1036484023 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f999406db30 0x7f99940a4fd0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f997c00d860 tx=0x7f997c008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:30.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.527+0000 7f998b7fe700 1 -- 172.21.15.110:0/1036484023 <== mon.2 v2:172.21.15.110:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997c0125f0 con 0x7f999406db30 2024-09-15T03:36:30.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.527+0000 7f998b7fe700 1 -- 172.21.15.110:0/1036484023 <== mon.2 v2:172.21.15.110:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f997c012750 con 0x7f999406db30 2024-09-15T03:36:30.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.527+0000 7f9999b98700 1 -- 172.21.15.110:0/1036484023 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99940a73a0 con 0x7f999406db30 2024-09-15T03:36:30.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.527+0000 7f998b7fe700 1 -- 172.21.15.110:0/1036484023 <== mon.2 v2:172.21.15.110:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997c012af0 con 0x7f999406db30 2024-09-15T03:36:30.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.527+0000 7f9999b98700 1 -- 172.21.15.110:0/1036484023 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99940a5510 con 0x7f999406db30 2024-09-15T03:36:30.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.527+0000 7f998b7fe700 1 -- 172.21.15.110:0/1036484023 <== mon.2 v2:172.21.15.110:3301/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f997c013710 con 0x7f999406db30 2024-09-15T03:36:30.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.527+0000 7f998b7fe700 1 -- 172.21.15.110:0/1036484023 <== mon.2 v2:172.21.15.110:3301/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7f997c01dbd0 con 0x7f999406db30 2024-09-15T03:36:30.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.527+0000 7f9999b98700 1 -- 172.21.15.110:0/1036484023 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f999411a510 con 0x7f999406db30 2024-09-15T03:36:30.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.531+0000 7f998b7fe700 1 -- 172.21.15.110:0/1036484023 <== mon.2 v2:172.21.15.110:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f997c013b00 con 0x7f999406db30 2024-09-15T03:36:30.596 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-15T03:36:30.591+0000 7fa4cdfbb000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-15T03:36:30.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.595+0000 7f9999b98700 1 -- 172.21.15.110:0/1036484023 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f99940a47b0 con 0x7f999406db30 2024-09-15T03:36:30.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.595+0000 7f998b7fe700 1 -- 172.21.15.110:0/1036484023 <== mon.2 v2:172.21.15.110:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7f99940a47b0 con 0x7f999406db30 2024-09-15T03:36:30.601 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-15T03:36:30.601 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":10,"fsid":"d7bdbc6e-e9f0-40e5-921a-f7c68fa2d8c1","created":"2024-09-15T03:36:28.375926+0000","modified":"2024-09-15T03:36:30.385381+0000","last_up_change":"0.000000","last_in_change":"2024-09-15T03:36:30.385381+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"09d93c1d-8ec1-42e8-8b59-5e20958830d6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"378acca0-6784-42c1-a6f8-0ba39a287db3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b1c2c060-985a-429c-bf05-ccfd5fdf3d8f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"aca55104-d76b-4719-8837-252c2e55d0cd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"7a87be3c-b6ff-43a4-8672-39d3006a0e2b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"a8f3a63d-45af-41fe-be33-2339d7b9ed96","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"a61b52dd-273d-47b4-bf92-0610cce86e1e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"1a7d5a9f-6b41-47e1-b0ef-45f4d638fd65","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T03:36:30.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.599+0000 7f9999b98700 1 -- 172.21.15.110:0/1036484023 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f999406db30 msgr2=0x7f99940a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:30.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.599+0000 7f9999b98700 1 --2- 172.21.15.110:0/1036484023 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f999406db30 0x7f99940a4fd0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f997c00d860 tx=0x7f997c008040 comp rx=0 tx=0).stop 2024-09-15T03:36:30.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.599+0000 7f9999b98700 1 -- 172.21.15.110:0/1036484023 shutdown_connections 2024-09-15T03:36:30.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.599+0000 7f9999b98700 1 --2- 172.21.15.110:0/1036484023 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f999406db30 0x7f99940a4fd0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:30.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.599+0000 7f9999b98700 1 --2- 172.21.15.110:0/1036484023 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f999406d180 0x7f99940a4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:30.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.599+0000 7f9999b98700 1 --2- 172.21.15.110:0/1036484023 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9994068ce0 0x7f9994118dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:30.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.599+0000 7f9999b98700 1 -- 172.21.15.110:0/1036484023 >> 172.21.15.110:0/1036484023 conn(0x7f99940b1150 msgr2=0x7f999413f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:30.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.599+0000 7f9999b98700 1 -- 172.21.15.110:0/1036484023 shutdown_connections 2024-09-15T03:36:30.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:30.599+0000 7f9999b98700 1 -- 172.21.15.110:0/1036484023 wait complete. 2024-09-15T03:36:30.607 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-15T03:36:30.607+0000 7febd9357000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-15T03:36:30.612 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-15T03:36:30.613 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-15T03:36:30.656 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-15T03:36:30.655+0000 7febd9357000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-15T03:36:30.763 INFO:tasks.ceph.osd.0.smithi110.stderr:2024-09-15T03:36:30.759+0000 7fd6657fd540 -1 Falling back to public interface 2024-09-15T03:36:30.777 INFO:tasks.ceph.osd.7.smithi177.stderr:2024-09-15T03:36:30.775+0000 7f0e7d218540 -1 Falling back to public interface 2024-09-15T03:36:30.777 INFO:tasks.ceph.osd.5.smithi177.stderr:2024-09-15T03:36:30.775+0000 7f2cf2323540 -1 Falling back to public interface 2024-09-15T03:36:30.779 INFO:tasks.ceph.osd.1.smithi110.stderr:2024-09-15T03:36:30.775+0000 7f62f0599540 -1 Falling back to public interface 2024-09-15T03:36:30.824 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-15T03:36:30.823+0000 7febd9357000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-15T03:36:30.882 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-15T03:36:30.879+0000 7febd9357000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-15T03:36:30.887 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-15T03:36:30.883+0000 7fa4cdfbb000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-15T03:36:30.943 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-15T03:36:30.939+0000 7febd9357000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-15T03:36:31.029 INFO:tasks.ceph.osd.6.smithi177.stderr:2024-09-15T03:36:31.027+0000 7fcd4e610540 -1 Falling back to public interface 2024-09-15T03:36:31.033 INFO:tasks.ceph.osd.4.smithi177.stderr:2024-09-15T03:36:31.031+0000 7f961068c540 -1 Falling back to public interface 2024-09-15T03:36:31.049 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-15T03:36:31.043+0000 7fa4cdfbb000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-15T03:36:31.051 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-15T03:36:31.047+0000 7fed6cc4e540 -1 Falling back to public interface 2024-09-15T03:36:31.055 INFO:tasks.ceph.osd.2.smithi110.stderr:2024-09-15T03:36:31.051+0000 7ff22c4c2540 -1 Falling back to public interface 2024-09-15T03:36:31.132 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-15T03:36:31.127+0000 7fa4cdfbb000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-15T03:36:31.188 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-15T03:36:31.183+0000 7fa4cdfbb000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-15T03:36:31.245 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-15T03:36:31.239+0000 7fa4cdfbb000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-15T03:36:31.335 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-15T03:36:31.335+0000 7febd9357000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-15T03:36:31.405 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-15T03:36:31.399+0000 7fa4cdfbb000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-15T03:36:31.417 DEBUG:teuthology.orchestra.run.smithi110:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-15T03:36:31.447 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-15T03:36:31.443+0000 7febd9357000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-15T03:36:31.512 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:31.507+0000 7fcaa70d7700 1 Processor -- start 2024-09-15T03:36:31.512 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:31.507+0000 7fcaa70d7700 1 -- start start 2024-09-15T03:36:31.513 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:31.507+0000 7fcaa70d7700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fcaa006f690 0x7fcaa006dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:31.513 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:31.507+0000 7fcaa70d7700 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaa006c480 con 0x7fcaa0070ec0 2024-09-15T03:36:31.513 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:31.507+0000 7fcaa70d7700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaa013b3c0 con 0x7fcaa013af70 2024-09-15T03:36:31.514 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:31.507+0000 7fcaa70d7700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaa006e270 con 0x7fcaa006f690 2024-09-15T03:36:31.514 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:31.507+0000 7fcaa4e73700 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fcaa0070ec0 0x7fcaa013a860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:53150/0 (socket says 172.21.15.110:53150) 2024-09-15T03:36:31.514 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:31.507+0000 7fcaa4e73700 1 -- 172.21.15.110:0/762292644 learned_addr learned my addr 172.21.15.110:0/762292644 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-15T03:36:31.514 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:31.507+0000 7fcaa5674700 1 --2- 172.21.15.110:0/762292644 >> v2:172.21.15.177:3300/0 conn(0x7fcaa006f690 0x7fcaa006dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:31.515 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:31.511+0000 7fca9f7fe700 1 -- 172.21.15.110:0/762292644 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3466934482 0 0) 0x7fcaa006c480 con 0x7fcaa0070ec0 2024-09-15T03:36:31.515 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:31.511+0000 7fca9f7fe700 1 -- 172.21.15.110:0/762292644 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca90003570 con 0x7fcaa0070ec0 2024-09-15T03:36:31.515 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:31.511+0000 7fcaa5674700 1 -- 172.21.15.110:0/762292644 >> v1:172.21.15.177:6789/0 conn(0x7fcaa013af70 legacy=0x7fcaa006d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:31.515 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:31.511+0000 7fcaa5674700 1 -- 172.21.15.110:0/762292644 >> v1:172.21.15.110:6789/0 conn(0x7fcaa0070ec0 legacy=0x7fcaa013a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:31.515 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:31.511+0000 7fcaa5674700 1 -- 172.21.15.110:0/762292644 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaa006ed20 con 0x7fcaa006f690 2024-09-15T03:36:31.515 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:31.511+0000 7fcaa5674700 1 --2- 172.21.15.110:0/762292644 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcaa006f690 0x7fcaa006dd30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fca98009e60 tx=0x7fca98009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db55af613f870828 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:31.515 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:31.511+0000 7fca9f7fe700 1 -- 172.21.15.110:0/762292644 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9800e070 con 0x7fcaa006f690 2024-09-15T03:36:31.515 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:31.511+0000 7fca9f7fe700 1 -- 172.21.15.110:0/762292644 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca9800bb60 con 0x7fcaa006f690 2024-09-15T03:36:31.516 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:31.511+0000 7fca9f7fe700 1 -- 172.21.15.110:0/762292644 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca98012430 con 0x7fcaa006f690 2024-09-15T03:36:31.516 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:31.511+0000 7fcaa70d7700 1 -- 172.21.15.110:0/762292644 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcaa006f690 msgr2=0x7fcaa006dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:31.516 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:31.511+0000 7fcaa70d7700 1 --2- 172.21.15.110:0/762292644 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcaa006f690 0x7fcaa006dd30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fca98009e60 tx=0x7fca98009780 comp rx=0 tx=0).stop 2024-09-15T03:36:31.516 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:31.511+0000 7fcaa70d7700 1 -- 172.21.15.110:0/762292644 shutdown_connections 2024-09-15T03:36:31.517 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:31.511+0000 7fcaa70d7700 1 --2- 172.21.15.110:0/762292644 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcaa006f690 0x7fcaa006dd30 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:31.517 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:31.511+0000 7fcaa70d7700 1 -- 172.21.15.110:0/762292644 >> 172.21.15.110:0/762292644 conn(0x7fcaa00b1420 msgr2=0x7fcaa00ad2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:31.517 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:31.511+0000 7fcaa70d7700 1 -- 172.21.15.110:0/762292644 shutdown_connections 2024-09-15T03:36:31.517 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:31.511+0000 7fcaa70d7700 1 -- 172.21.15.110:0/762292644 wait complete. 2024-09-15T03:36:31.519 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:31.511+0000 7fcaa70d7700 1 Processor -- start 2024-09-15T03:36:31.519 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:31.511+0000 7fcaa70d7700 1 -- start start 2024-09-15T03:36:31.519 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:31.515+0000 7fcaa70d7700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcaa006f690 0x7fcaa013e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:31.519 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:31.515+0000 7fcaa70d7700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcaa0070ec0 0x7fcaa01375b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:31.519 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:31.515+0000 7fcaa70d7700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fcaa013af70 0x7fcaa0134ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:31.519 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:31.515+0000 7fcaa70d7700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa01370d0 con 0x7fcaa0070ec0 2024-09-15T03:36:31.520 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:31.515+0000 7fcaa70d7700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa0135f90 con 0x7fcaa006f690 2024-09-15T03:36:31.520 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:31.515+0000 7fcaa70d7700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa0136b60 con 0x7fcaa013af70 2024-09-15T03:36:31.520 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:31.515+0000 7fca9ffff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcaa0070ec0 0x7fcaa01375b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:31.520 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:31.515+0000 7fca9ffff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcaa0070ec0 0x7fcaa01375b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52980/0 (socket says 172.21.15.110:52980) 2024-09-15T03:36:31.520 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:31.515+0000 7fcaa5674700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fcaa013af70 0x7fcaa0134ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:31.520 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:31.515+0000 7fca9ffff700 1 -- 172.21.15.110:0/736354606 learned_addr learned my addr 172.21.15.110:0/736354606 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:36:31.520 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:31.515+0000 7fcaa4e73700 1 --2- 172.21.15.110:0/736354606 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcaa006f690 0x7fcaa013e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:31.520 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:31.515+0000 7fca9ffff700 1 -- 172.21.15.110:0/736354606 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fcaa013af70 msgr2=0x7fcaa0134ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:31.520 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:31.515+0000 7fca9ffff700 1 --2- 172.21.15.110:0/736354606 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fcaa013af70 0x7fcaa0134ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:31.520 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:31.515+0000 7fca9ffff700 1 -- 172.21.15.110:0/736354606 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcaa006f690 msgr2=0x7fcaa013e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:31.520 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:31.515+0000 7fca9ffff700 1 --2- 172.21.15.110:0/736354606 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcaa006f690 0x7fcaa013e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:31.520 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:31.515+0000 7fca9ffff700 1 -- 172.21.15.110:0/736354606 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca98009a20 con 0x7fcaa0070ec0 2024-09-15T03:36:31.521 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:31.515+0000 7fca9ffff700 1 --2- 172.21.15.110:0/736354606 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcaa0070ec0 0x7fcaa01375b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fca8c00e9d0 tx=0x7fca8c00c300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:31.521 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:31.515+0000 7fca9cff9700 1 -- 172.21.15.110:0/736354606 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8c0106e0 con 0x7fcaa0070ec0 2024-09-15T03:36:31.521 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:31.515+0000 7fca9cff9700 1 -- 172.21.15.110:0/736354606 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca8c010840 con 0x7fcaa0070ec0 2024-09-15T03:36:31.521 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:31.515+0000 7fca9cff9700 1 -- 172.21.15.110:0/736354606 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8c010c00 con 0x7fcaa0070ec0 2024-09-15T03:36:31.521 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:31.515+0000 7fcaa70d7700 1 -- 172.21.15.110:0/736354606 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaa0136d80 con 0x7fcaa0070ec0 2024-09-15T03:36:31.521 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:31.515+0000 7fcaa70d7700 1 -- 172.21.15.110:0/736354606 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaa00a46d0 con 0x7fcaa0070ec0 2024-09-15T03:36:31.521 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:31.515+0000 7fca9cff9700 1 -- 172.21.15.110:0/736354606 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fca8c017030 con 0x7fcaa0070ec0 2024-09-15T03:36:31.521 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:31.515+0000 7fca9cff9700 1 -- 172.21.15.110:0/736354606 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7fca8c015070 con 0x7fcaa0070ec0 2024-09-15T03:36:31.522 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:31.515+0000 7fcaa70d7700 1 -- 172.21.15.110:0/736354606 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaa0137280 con 0x7fcaa0070ec0 2024-09-15T03:36:31.523 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:31.519+0000 7fca9cff9700 1 -- 172.21.15.110:0/736354606 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7fca8c01f060 con 0x7fcaa0070ec0 2024-09-15T03:36:31.572 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-15T03:36:31.567+0000 7fa4cdfbb000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-15T03:36:31.587 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:31.583+0000 7fcaa70d7700 1 -- 172.21.15.110:0/736354606 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcaa0064d80 con 0x7fcaa0070ec0 2024-09-15T03:36:31.588 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:31.583+0000 7fca9cff9700 1 -- 172.21.15.110:0/736354606 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7fca8c029090 con 0x7fcaa0070ec0 2024-09-15T03:36:31.589 INFO:teuthology.misc.health.smithi110.stdout: 2024-09-15T03:36:31.589 INFO:teuthology.misc.health.smithi110.stdout:{"epoch":10,"fsid":"d7bdbc6e-e9f0-40e5-921a-f7c68fa2d8c1","created":"2024-09-15T03:36:28.375926+0000","modified":"2024-09-15T03:36:30.385381+0000","last_up_change":"0.000000","last_in_change":"2024-09-15T03:36:30.385381+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"09d93c1d-8ec1-42e8-8b59-5e20958830d6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"378acca0-6784-42c1-a6f8-0ba39a287db3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b1c2c060-985a-429c-bf05-ccfd5fdf3d8f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"aca55104-d76b-4719-8837-252c2e55d0cd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"7a87be3c-b6ff-43a4-8672-39d3006a0e2b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"a8f3a63d-45af-41fe-be33-2339d7b9ed96","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"a61b52dd-273d-47b4-bf92-0610cce86e1e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"1a7d5a9f-6b41-47e1-b0ef-45f4d638fd65","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T03:36:31.590 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:31.583+0000 7fcaa70d7700 1 -- 172.21.15.110:0/736354606 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcaa0070ec0 msgr2=0x7fcaa01375b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:31.590 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:31.583+0000 7fcaa70d7700 1 --2- 172.21.15.110:0/736354606 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcaa0070ec0 0x7fcaa01375b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fca8c00e9d0 tx=0x7fca8c00c300 comp rx=0 tx=0).stop 2024-09-15T03:36:31.590 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:31.583+0000 7fcaa70d7700 1 -- 172.21.15.110:0/736354606 shutdown_connections 2024-09-15T03:36:31.590 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:31.583+0000 7fcaa70d7700 1 --2- 172.21.15.110:0/736354606 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fcaa013af70 0x7fcaa0134ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:31.590 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:31.583+0000 7fcaa70d7700 1 --2- 172.21.15.110:0/736354606 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcaa0070ec0 0x7fcaa01375b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:31.590 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:31.583+0000 7fcaa70d7700 1 --2- 172.21.15.110:0/736354606 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcaa006f690 0x7fcaa013e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:31.590 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:31.583+0000 7fcaa70d7700 1 -- 172.21.15.110:0/736354606 >> 172.21.15.110:0/736354606 conn(0x7fcaa00b1420 msgr2=0x7fcaa0064120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:31.590 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:31.587+0000 7fcaa70d7700 1 -- 172.21.15.110:0/736354606 shutdown_connections 2024-09-15T03:36:31.590 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:31.587+0000 7fcaa70d7700 1 -- 172.21.15.110:0/736354606 wait complete. 2024-09-15T03:36:31.599 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-15T03:36:31.639 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-15T03:36:31.635+0000 7fa4cdfbb000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-15T03:36:31.736 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-15T03:36:31.735+0000 7febd9357000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-15T03:36:31.798 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-15T03:36:31.795+0000 7fa4cdfbb000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-15T03:36:31.867 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-15T03:36:31.863+0000 7fa4cdfbb000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-15T03:36:31.897 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-15T03:36:31.895+0000 7febd9357000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-15T03:36:31.964 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-15T03:36:31.963+0000 7febd9357000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-15T03:36:32.016 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-15T03:36:32.015+0000 7febd9357000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-15T03:36:32.073 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-15T03:36:32.071+0000 7febd9357000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-15T03:36:32.232 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-15T03:36:32.231+0000 7febd9357000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-15T03:36:32.401 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-15T03:36:32.399+0000 7febd9357000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-15T03:36:32.467 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-15T03:36:32.467+0000 7febd9357000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-15T03:36:32.573 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-15T03:36:32.567+0000 7fa4cdfbb000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-15T03:36:32.627 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-15T03:36:32.627+0000 7febd9357000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-15T03:36:32.697 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-15T03:36:32.695+0000 7febd9357000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-15T03:36:33.400 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-09-15T03:36:33.399+0000 7febd9357000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-15T03:36:33.615 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-15T03:36:33.611+0000 7fa4b702f700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-15T03:36:33.616 INFO:tasks.ceph.mgr.y.smithi110.stderr:2024-09-15T03:36:33.611+0000 7fa4b702f700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-15T03:36:34.106 INFO:tasks.ceph.osd.0.smithi110.stderr:2024-09-15T03:36:34.103+0000 7fd6657fd540 -1 osd.0 0 log_to_monitors true 2024-09-15T03:36:34.531 INFO:tasks.ceph.osd.1.smithi110.stderr:2024-09-15T03:36:34.527+0000 7f62f0599540 -1 osd.1 0 log_to_monitors true 2024-09-15T03:36:34.597 INFO:tasks.ceph.osd.4.smithi177.stderr:2024-09-15T03:36:34.595+0000 7f961068c540 -1 osd.4 0 log_to_monitors true 2024-09-15T03:36:34.608 INFO:tasks.ceph.osd.6.smithi177.stderr:2024-09-15T03:36:34.607+0000 7fcd4e610540 -1 osd.6 0 log_to_monitors true 2024-09-15T03:36:34.623 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-15T03:36:34.619+0000 7fed6cc4e540 -1 osd.3 0 log_to_monitors true 2024-09-15T03:36:34.647 INFO:tasks.ceph.osd.7.smithi177.stderr:2024-09-15T03:36:34.647+0000 7f0e7d218540 -1 osd.7 0 log_to_monitors true 2024-09-15T03:36:34.674 INFO:tasks.ceph.osd.5.smithi177.stderr:2024-09-15T03:36:34.671+0000 7f2cf2323540 -1 osd.5 0 log_to_monitors true 2024-09-15T03:36:34.868 INFO:tasks.ceph.osd.2.smithi110.stderr:2024-09-15T03:36:34.863+0000 7ff22c4c2540 -1 osd.2 0 log_to_monitors true 2024-09-15T03:36:35.628 INFO:tasks.ceph.osd.4.smithi177.stderr:2024-09-15T03:36:35.615+0000 7f960be2c700 -1 osd.4 0 waiting for initial osdmap 2024-09-15T03:36:35.628 INFO:tasks.ceph.osd.0.smithi110.stderr:2024-09-15T03:36:35.619+0000 7fd660f9d700 -1 osd.0 0 waiting for initial osdmap 2024-09-15T03:36:35.629 INFO:tasks.ceph.osd.1.smithi110.stderr:2024-09-15T03:36:35.619+0000 7f62ebd39700 -1 osd.1 0 waiting for initial osdmap 2024-09-15T03:36:35.630 INFO:tasks.ceph.osd.4.smithi177.stderr:2024-09-15T03:36:35.627+0000 7f96063d2700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T03:36:35.630 INFO:tasks.ceph.osd.0.smithi110.stderr:2024-09-15T03:36:35.623+0000 7fd65c545700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T03:36:35.632 INFO:tasks.ceph.osd.1.smithi110.stderr:2024-09-15T03:36:35.627+0000 7f62e62df700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T03:36:36.622 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-15T03:36:36.615+0000 7fed683ee700 -1 osd.3 0 waiting for initial osdmap 2024-09-15T03:36:36.622 INFO:tasks.ceph.osd.6.smithi177.stderr:2024-09-15T03:36:36.619+0000 7fcd4b5d5700 -1 osd.6 0 waiting for initial osdmap 2024-09-15T03:36:36.623 INFO:tasks.ceph.osd.7.smithi177.stderr:2024-09-15T03:36:36.623+0000 7f0e789b8700 -1 osd.7 0 waiting for initial osdmap 2024-09-15T03:36:36.623 INFO:tasks.ceph.osd.5.smithi177.stderr:2024-09-15T03:36:36.623+0000 7f2cef2e8700 -1 osd.5 0 waiting for initial osdmap 2024-09-15T03:36:36.624 INFO:tasks.ceph.osd.2.smithi110.stderr:2024-09-15T03:36:36.619+0000 7ff227c62700 -1 osd.2 0 waiting for initial osdmap 2024-09-15T03:36:36.629 INFO:tasks.ceph.osd.2.smithi110.stderr:2024-09-15T03:36:36.623+0000 7ff222208700 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T03:36:36.629 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-15T03:36:36.623+0000 7fed63195700 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T03:36:36.629 INFO:tasks.ceph.osd.6.smithi177.stderr:2024-09-15T03:36:36.627+0000 7fcd44b57700 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T03:36:36.630 INFO:tasks.ceph.osd.7.smithi177.stderr:2024-09-15T03:36:36.627+0000 7f0e72f5e700 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T03:36:36.632 INFO:tasks.ceph.osd.5.smithi177.stderr:2024-09-15T03:36:36.631+0000 7f2ce886a700 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T03:36:38.406 DEBUG:teuthology.orchestra.run.smithi110:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-15T03:36:38.505 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.499+0000 7f165f59e700 1 Processor -- start 2024-09-15T03:36:38.506 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.499+0000 7f165f59e700 1 -- start start 2024-09-15T03:36:38.507 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.503+0000 7f165f59e700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f16601416d0 0x7f166006dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:38.507 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.503+0000 7f165f59e700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f166006e970 0x7f166006edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:38.507 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.503+0000 7f165f59e700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1660141240 con 0x7f16600ad470 2024-09-15T03:36:38.507 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.503+0000 7f165f59e700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16600adf50 con 0x7f16601416d0 2024-09-15T03:36:38.507 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.503+0000 7f165f59e700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f166006f330 con 0x7f166006e970 2024-09-15T03:36:38.507 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.503+0000 7f165ed9d700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f166006e970 0x7f166006edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:38.507 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.503+0000 7f165ed9d700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f166006e970 0x7f166006edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3301/0 says I am v2:172.21.15.110:44560/0 (socket says 172.21.15.110:44560) 2024-09-15T03:36:38.508 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.503+0000 7f165ed9d700 1 -- 172.21.15.110:0/366109494 learned_addr learned my addr 172.21.15.110:0/366109494 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:36:38.508 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.503+0000 7f1655d9b700 1 --2- 172.21.15.110:0/366109494 >> v2:172.21.15.177:3300/0 conn(0x7f16601416d0 0x7f166006dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:38.508 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.503+0000 7f165ed9d700 1 -- 172.21.15.110:0/366109494 >> v2:172.21.15.177:3300/0 conn(0x7f16601416d0 msgr2=0x7f166006dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:36:38.508 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.503+0000 7f165ed9d700 1 --2- 172.21.15.110:0/366109494 >> v2:172.21.15.177:3300/0 conn(0x7f16601416d0 0x7f166006dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:38.508 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.503+0000 7f165ed9d700 1 -- 172.21.15.110:0/366109494 >> v1:172.21.15.177:6789/0 conn(0x7f16600ad470 legacy=0x7f16600ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:38.508 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.503+0000 7f165ed9d700 1 -- 172.21.15.110:0/366109494 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1660141ac0 con 0x7f166006e970 2024-09-15T03:36:38.508 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.503+0000 7f165ed9d700 1 --2- 172.21.15.110:0/366109494 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f166006e970 0x7f166006edf0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1658009f20 tx=0x7f1658009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f8b292e744d6702 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:38.509 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.503+0000 7f165dd9b700 1 -- 172.21.15.110:0/366109494 <== mon.2 v2:172.21.15.110:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f165800e070 con 0x7f166006e970 2024-09-15T03:36:38.509 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.503+0000 7f165dd9b700 1 -- 172.21.15.110:0/366109494 <== mon.2 v2:172.21.15.110:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f165800b9d0 con 0x7f166006e970 2024-09-15T03:36:38.509 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.503+0000 7f165dd9b700 1 -- 172.21.15.110:0/366109494 <== mon.2 v2:172.21.15.110:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16580092e0 con 0x7f166006e970 2024-09-15T03:36:38.510 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.503+0000 7f165f59e700 1 -- 172.21.15.110:0/366109494 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f166006e970 msgr2=0x7f166006edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:38.510 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.503+0000 7f165f59e700 1 --2- 172.21.15.110:0/366109494 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f166006e970 0x7f166006edf0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1658009f20 tx=0x7f1658009840 comp rx=0 tx=0).stop 2024-09-15T03:36:38.510 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.503+0000 7f165f59e700 1 -- 172.21.15.110:0/366109494 shutdown_connections 2024-09-15T03:36:38.510 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.503+0000 7f165f59e700 1 --2- 172.21.15.110:0/366109494 >> v2:172.21.15.177:3300/0 conn(0x7f16601416d0 0x7f166006dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:38.510 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.503+0000 7f165f59e700 1 --2- 172.21.15.110:0/366109494 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f166006e970 0x7f166006edf0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:38.510 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.503+0000 7f165f59e700 1 -- 172.21.15.110:0/366109494 >> 172.21.15.110:0/366109494 conn(0x7f16600b21b0 msgr2=0x7f16600b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:38.510 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.503+0000 7f165f59e700 1 -- 172.21.15.110:0/366109494 shutdown_connections 2024-09-15T03:36:38.510 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.503+0000 7f165f59e700 1 -- 172.21.15.110:0/366109494 wait complete. 2024-09-15T03:36:38.511 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.507+0000 7f165f59e700 1 Processor -- start 2024-09-15T03:36:38.511 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.507+0000 7f165f59e700 1 -- start start 2024-09-15T03:36:38.512 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.507+0000 7f165f59e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f166006e970 0x7f1660141d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:38.512 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.507+0000 7f165f59e700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f16600ad470 0x7f166013dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:38.512 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.507+0000 7f165f59e700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16601416d0 0x7f1660144530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:38.512 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.507+0000 7f165f59e700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f166013cf70 con 0x7f16601416d0 2024-09-15T03:36:38.512 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.507+0000 7f165f59e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1660144a70 con 0x7f166006e970 2024-09-15T03:36:38.512 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.507+0000 7f165f59e700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f166013b490 con 0x7f16600ad470 2024-09-15T03:36:38.512 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.507+0000 7f1655d9b700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f16600ad470 0x7f166013dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:38.512 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.507+0000 7f1655d9b700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f16600ad470 0x7f166013dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3301/0 says I am v2:172.21.15.110:44562/0 (socket says 172.21.15.110:44562) 2024-09-15T03:36:38.512 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.507+0000 7f1655d9b700 1 -- 172.21.15.110:0/3567657406 learned_addr learned my addr 172.21.15.110:0/3567657406 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:36:38.513 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.507+0000 7f165ed9d700 1 --2- 172.21.15.110:0/3567657406 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16601416d0 0x7f1660144530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:38.513 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.507+0000 7f165e59c700 1 --2- 172.21.15.110:0/3567657406 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f166006e970 0x7f1660141d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:38.515 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.507+0000 7f165ed9d700 1 -- 172.21.15.110:0/3567657406 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f16600ad470 msgr2=0x7f166013dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:38.515 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.507+0000 7f165ed9d700 1 --2- 172.21.15.110:0/3567657406 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f16600ad470 0x7f166013dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:38.515 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.507+0000 7f165ed9d700 1 -- 172.21.15.110:0/3567657406 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f166006e970 msgr2=0x7f1660141d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:38.515 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.507+0000 7f165ed9d700 1 --2- 172.21.15.110:0/3567657406 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f166006e970 0x7f1660141d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:38.515 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.507+0000 7f165ed9d700 1 -- 172.21.15.110:0/3567657406 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1658009ae0 con 0x7f16601416d0 2024-09-15T03:36:38.515 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.507+0000 7f1655d9b700 1 --2- 172.21.15.110:0/3567657406 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f16600ad470 0x7f166013dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T03:36:38.516 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.507+0000 7f165ed9d700 1 --2- 172.21.15.110:0/3567657406 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16601416d0 0x7f1660144530 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1658015000 tx=0x7f1658018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:38.516 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.507+0000 7f1656ffd700 1 -- 172.21.15.110:0/3567657406 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1658008ad0 con 0x7f16601416d0 2024-09-15T03:36:38.516 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.507+0000 7f1656ffd700 1 -- 172.21.15.110:0/3567657406 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1658008c50 con 0x7f16601416d0 2024-09-15T03:36:38.516 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.507+0000 7f165f59e700 1 -- 172.21.15.110:0/3567657406 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f166013d170 con 0x7f16601416d0 2024-09-15T03:36:38.516 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.507+0000 7f1656ffd700 1 -- 172.21.15.110:0/3567657406 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1658025460 con 0x7f16601416d0 2024-09-15T03:36:38.516 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.507+0000 7f165f59e700 1 -- 172.21.15.110:0/3567657406 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f166013be90 con 0x7f16601416d0 2024-09-15T03:36:38.518 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.511+0000 7f165f59e700 1 -- 172.21.15.110:0/3567657406 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1660064d80 con 0x7f16601416d0 2024-09-15T03:36:38.518 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.511+0000 7f1656ffd700 1 -- 172.21.15.110:0/3567657406 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 4) v1 ==== 85791+0+0 (secure 0 0 0) 0x7f1658003040 con 0x7f16601416d0 2024-09-15T03:36:38.518 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.511+0000 7f1656ffd700 1 --2- 172.21.15.110:0/3567657406 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f16440679f0 0x7f1644069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:38.518 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.511+0000 7f1656ffd700 1 -- 172.21.15.110:0/3567657406 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4842+0+0 (secure 0 0 0) 0x7f16580867d0 con 0x7f16601416d0 2024-09-15T03:36:38.518 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.511+0000 7f165e59c700 1 --2- 172.21.15.110:0/3567657406 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f16440679f0 0x7f1644069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:38.519 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.511+0000 7f165e59c700 1 --2- 172.21.15.110:0/3567657406 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f16440679f0 0x7f1644069ea0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1650003a80 tx=0x7f1650007ec0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:38.528 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.523+0000 7f1656ffd700 1 -- 172.21.15.110:0/3567657406 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f1658053d10 con 0x7f16601416d0 2024-09-15T03:36:38.693 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.687+0000 7f165f59e700 1 -- 172.21.15.110:0/3567657406 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f166013abf0 con 0x7f16601416d0 2024-09-15T03:36:38.694 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.691+0000 7f1656ffd700 1 -- 172.21.15.110:0/3567657406 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+12188 (secure 0 0 0) 0x7f1658056d90 con 0x7f16601416d0 2024-09-15T03:36:38.694 INFO:teuthology.misc.health.smithi110.stdout: 2024-09-15T03:36:38.694 INFO:teuthology.misc.health.smithi110.stdout:{"epoch":15,"fsid":"d7bdbc6e-e9f0-40e5-921a-f7c68fa2d8c1","created":"2024-09-15T03:36:28.375926+0000","modified":"2024-09-15T03:36:38.630682+0000","last_up_change":"2024-09-15T03:36:37.618040+0000","last_in_change":"2024-09-15T03:36:30.385381+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T03:36:36.657583+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"09d93c1d-8ec1-42e8-8b59-5e20958830d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":22352},{"type":"v1","addr":"172.21.15.110:6801","nonce":22352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":22352},{"type":"v1","addr":"172.21.15.110:6803","nonce":22352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":22352},{"type":"v1","addr":"172.21.15.110:6807","nonce":22352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":22352},{"type":"v1","addr":"172.21.15.110:6805","nonce":22352}]},"public_addr":"172.21.15.110:6801/22352","cluster_addr":"172.21.15.110:6803/22352","heartbeat_back_addr":"172.21.15.110:6807/22352","heartbeat_front_addr":"172.21.15.110:6805/22352","state":["exists","up"]},{"osd":1,"uuid":"378acca0-6784-42c1-a6f8-0ba39a287db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":22354},{"type":"v1","addr":"172.21.15.110:6809","nonce":22354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":22354},{"type":"v1","addr":"172.21.15.110:6811","nonce":22354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":22354},{"type":"v1","addr":"172.21.15.110:6815","nonce":22354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":22354},{"type":"v1","addr":"172.21.15.110:6813","nonce":22354}]},"public_addr":"172.21.15.110:6809/22354","cluster_addr":"172.21.15.110:6811/22354","heartbeat_back_addr":"172.21.15.110:6815/22354","heartbeat_front_addr":"172.21.15.110:6813/22354","state":["exists","up"]},{"osd":2,"uuid":"b1c2c060-985a-429c-bf05-ccfd5fdf3d8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":22356},{"type":"v1","addr":"172.21.15.110:6824","nonce":22356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":22356},{"type":"v1","addr":"172.21.15.110:6827","nonce":22356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":22356},{"type":"v1","addr":"172.21.15.110:6831","nonce":22356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":22356},{"type":"v1","addr":"172.21.15.110:6829","nonce":22356}]},"public_addr":"172.21.15.110:6824/22356","cluster_addr":"172.21.15.110:6827/22356","heartbeat_back_addr":"172.21.15.110:6831/22356","heartbeat_front_addr":"172.21.15.110:6829/22356","state":["exists","up"]},{"osd":3,"uuid":"aca55104-d76b-4719-8837-252c2e55d0cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":22355},{"type":"v1","addr":"172.21.15.110:6817","nonce":22355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":22355},{"type":"v1","addr":"172.21.15.110:6819","nonce":22355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6823","nonce":22355},{"type":"v1","addr":"172.21.15.110:6825","nonce":22355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":22355},{"type":"v1","addr":"172.21.15.110:6821","nonce":22355}]},"public_addr":"172.21.15.110:6817/22355","cluster_addr":"172.21.15.110:6819/22355","heartbeat_back_addr":"172.21.15.110:6825/22355","heartbeat_front_addr":"172.21.15.110:6821/22355","state":["exists","up"]},{"osd":4,"uuid":"7a87be3c-b6ff-43a4-8672-39d3006a0e2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":22425},{"type":"v1","addr":"172.21.15.177:6824","nonce":22425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":22425},{"type":"v1","addr":"172.21.15.177:6827","nonce":22425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":22425},{"type":"v1","addr":"172.21.15.177:6831","nonce":22425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":22425},{"type":"v1","addr":"172.21.15.177:6829","nonce":22425}]},"public_addr":"172.21.15.177:6824/22425","cluster_addr":"172.21.15.177:6827/22425","heartbeat_back_addr":"172.21.15.177:6831/22425","heartbeat_front_addr":"172.21.15.177:6829/22425","state":["exists","up"]},{"osd":5,"uuid":"a8f3a63d-45af-41fe-be33-2339d7b9ed96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":22424},{"type":"v1","addr":"172.21.15.177:6804","nonce":22424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":22424},{"type":"v1","addr":"172.21.15.177:6808","nonce":22424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":22424},{"type":"v1","addr":"172.21.15.177:6815","nonce":22424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":22424},{"type":"v1","addr":"172.21.15.177:6812","nonce":22424}]},"public_addr":"172.21.15.177:6804/22424","cluster_addr":"172.21.15.177:6808/22424","heartbeat_back_addr":"172.21.15.177:6815/22424","heartbeat_front_addr":"172.21.15.177:6812/22424","state":["exists","up"]},{"osd":6,"uuid":"a61b52dd-273d-47b4-bf92-0610cce86e1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":22428},{"type":"v1","addr":"172.21.15.177:6817","nonce":22428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":22428},{"type":"v1","addr":"172.21.15.177:6819","nonce":22428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":22428},{"type":"v1","addr":"172.21.15.177:6825","nonce":22428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":22428},{"type":"v1","addr":"172.21.15.177:6821","nonce":22428}]},"public_addr":"172.21.15.177:6817/22428","cluster_addr":"172.21.15.177:6819/22428","heartbeat_back_addr":"172.21.15.177:6825/22428","heartbeat_front_addr":"172.21.15.177:6821/22428","state":["exists","up"]},{"osd":7,"uuid":"1a7d5a9f-6b41-47e1-b0ef-45f4d638fd65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":22423},{"type":"v1","addr":"172.21.15.177:6801","nonce":22423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6803","nonce":22423},{"type":"v1","addr":"172.21.15.177:6805","nonce":22423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6811","nonce":22423},{"type":"v1","addr":"172.21.15.177:6813","nonce":22423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6807","nonce":22423},{"type":"v1","addr":"172.21.15.177:6809","nonce":22423}]},"public_addr":"172.21.15.177:6801/22423","cluster_addr":"172.21.15.177:6805/22423","heartbeat_back_addr":"172.21.15.177:6813/22423","heartbeat_front_addr":"172.21.15.177:6809/22423","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:36:35.106295+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:36:35.495899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:36:35.902086+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:36:35.652162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:36:35.645490+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:36:35.658420+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:36:35.622386+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:36:35.604615+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T03:36:38.697 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.691+0000 7f165f59e700 1 -- 172.21.15.110:0/3567657406 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f16440679f0 msgr2=0x7f1644069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:38.697 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.691+0000 7f165f59e700 1 --2- 172.21.15.110:0/3567657406 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f16440679f0 0x7f1644069ea0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1650003a80 tx=0x7f1650007ec0 comp rx=0 tx=0).stop 2024-09-15T03:36:38.697 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.691+0000 7f165f59e700 1 -- 172.21.15.110:0/3567657406 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16601416d0 msgr2=0x7f1660144530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:38.697 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.691+0000 7f165f59e700 1 --2- 172.21.15.110:0/3567657406 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16601416d0 0x7f1660144530 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1658015000 tx=0x7f1658018000 comp rx=0 tx=0).stop 2024-09-15T03:36:38.697 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.691+0000 7f165f59e700 1 -- 172.21.15.110:0/3567657406 shutdown_connections 2024-09-15T03:36:38.698 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.691+0000 7f165f59e700 1 --2- 172.21.15.110:0/3567657406 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f16440679f0 0x7f1644069ea0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:38.698 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.691+0000 7f165f59e700 1 --2- 172.21.15.110:0/3567657406 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16601416d0 0x7f1660144530 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:38.698 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.691+0000 7f165f59e700 1 --2- 172.21.15.110:0/3567657406 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f16600ad470 0x7f166013dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:38.698 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.691+0000 7f165f59e700 1 --2- 172.21.15.110:0/3567657406 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f166006e970 0x7f1660141d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:38.698 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.691+0000 7f165f59e700 1 -- 172.21.15.110:0/3567657406 >> 172.21.15.110:0/3567657406 conn(0x7f16600b21b0 msgr2=0x7f16600ad840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:38.698 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.691+0000 7f165f59e700 1 -- 172.21.15.110:0/3567657406 shutdown_connections 2024-09-15T03:36:38.698 INFO:teuthology.misc.health.smithi110.stderr:2024-09-15T03:36:38.691+0000 7f165f59e700 1 -- 172.21.15.110:0/3567657406 wait complete. 2024-09-15T03:36:38.708 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-15T03:36:38.708 INFO:tasks.ceph:Creating RBD pool 2024-09-15T03:36:38.708 DEBUG:teuthology.orchestra.run.smithi110:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-15T03:36:38.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:38.807+0000 7f7536cb6700 1 Processor -- start 2024-09-15T03:36:38.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:38.807+0000 7f7536cb6700 1 -- start start 2024-09-15T03:36:38.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:38.807+0000 7f7536cb6700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f7530068680 0x7f75300ac080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:38.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:38.807+0000 7f7536cb6700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f7530137a70 0x7f7530144c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:38.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:38.807+0000 7f7536cb6700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f753013e7c0 con 0x7f75300ac5c0 2024-09-15T03:36:38.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:38.807+0000 7f7536cb6700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f753006eac0 con 0x7f7530137a70 2024-09-15T03:36:38.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:38.807+0000 7f7536cb6700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f753006f200 con 0x7f7530068680 2024-09-15T03:36:38.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:38.807+0000 7f7535cb4700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f7530068680 0x7f75300ac080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:38.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:38.807+0000 7f7535cb4700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f7530068680 0x7f75300ac080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3301/0 says I am v2:172.21.15.110:44578/0 (socket says 172.21.15.110:44578) 2024-09-15T03:36:38.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:38.807+0000 7f7535cb4700 1 -- 172.21.15.110:0/1644068706 learned_addr learned my addr 172.21.15.110:0/1644068706 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:36:38.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:38.807+0000 7f75364b5700 1 --2- 172.21.15.110:0/1644068706 >> v2:172.21.15.177:3300/0 conn(0x7f7530137a70 0x7f7530144c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:38.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:38.807+0000 7f7535cb4700 1 -- 172.21.15.110:0/1644068706 >> v2:172.21.15.177:3300/0 conn(0x7f7530137a70 msgr2=0x7f7530144c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:36:38.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:38.807+0000 7f7535cb4700 1 --2- 172.21.15.110:0/1644068706 >> v2:172.21.15.177:3300/0 conn(0x7f7530137a70 0x7f7530144c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:38.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:38.807+0000 7f7535cb4700 1 -- 172.21.15.110:0/1644068706 >> v1:172.21.15.177:6789/0 conn(0x7f75300ac5c0 legacy=0x7f7530137360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:38.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:38.807+0000 7f7535cb4700 1 -- 172.21.15.110:0/1644068706 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f753006e0a0 con 0x7f7530068680 2024-09-15T03:36:38.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:38.807+0000 7f7535cb4700 1 --2- 172.21.15.110:0/1644068706 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f7530068680 0x7f75300ac080 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7528009f20 tx=0x7f7528009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fcb993920437bdc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:38.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:38.807+0000 7f7534cb2700 1 -- 172.21.15.110:0/1644068706 <== mon.2 v2:172.21.15.110:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752800e070 con 0x7f7530068680 2024-09-15T03:36:38.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:38.807+0000 7f7534cb2700 1 -- 172.21.15.110:0/1644068706 <== mon.2 v2:172.21.15.110:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f752800b9d0 con 0x7f7530068680 2024-09-15T03:36:38.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:38.807+0000 7f7534cb2700 1 -- 172.21.15.110:0/1644068706 <== mon.2 v2:172.21.15.110:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75280092e0 con 0x7f7530068680 2024-09-15T03:36:38.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:38.811+0000 7f7536cb6700 1 -- 172.21.15.110:0/1644068706 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f7530068680 msgr2=0x7f75300ac080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:38.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:38.811+0000 7f7536cb6700 1 --2- 172.21.15.110:0/1644068706 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f7530068680 0x7f75300ac080 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7528009f20 tx=0x7f7528009840 comp rx=0 tx=0).stop 2024-09-15T03:36:38.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:38.811+0000 7f7536cb6700 1 -- 172.21.15.110:0/1644068706 shutdown_connections 2024-09-15T03:36:38.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:38.811+0000 7f7536cb6700 1 --2- 172.21.15.110:0/1644068706 >> v2:172.21.15.177:3300/0 conn(0x7f7530137a70 0x7f7530144c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:38.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:38.811+0000 7f7536cb6700 1 --2- 172.21.15.110:0/1644068706 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f7530068680 0x7f75300ac080 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:38.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:38.811+0000 7f7536cb6700 1 -- 172.21.15.110:0/1644068706 >> 172.21.15.110:0/1644068706 conn(0x7f75300b1040 msgr2=0x7f75300ad070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:38.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:38.811+0000 7f7536cb6700 1 -- 172.21.15.110:0/1644068706 shutdown_connections 2024-09-15T03:36:38.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:38.811+0000 7f7536cb6700 1 -- 172.21.15.110:0/1644068706 wait complete. 2024-09-15T03:36:38.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:38.811+0000 7f7536cb6700 1 Processor -- start 2024-09-15T03:36:38.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:38.811+0000 7f7536cb6700 1 -- start start 2024-09-15T03:36:38.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:38.811+0000 7f7536cb6700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7530068680 0x7f753013a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:38.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:38.811+0000 7f7536cb6700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f75300ac5c0 0x7f753013d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:38.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:38.811+0000 7f7536cb6700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f7530137a70 0x7f753014e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:38.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:38.811+0000 7f7536cb6700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f753013d950 con 0x7f7530068680 2024-09-15T03:36:38.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:38.811+0000 7f7536cb6700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f753013b1c0 con 0x7f75300ac5c0 2024-09-15T03:36:38.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:38.811+0000 7f7536cb6700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75300a4e80 con 0x7f7530137a70 2024-09-15T03:36:38.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:38.811+0000 7f75364b5700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f7530137a70 0x7f753014e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:38.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:38.811+0000 7f75364b5700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f7530137a70 0x7f753014e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3301/0 says I am v2:172.21.15.110:44586/0 (socket says 172.21.15.110:44586) 2024-09-15T03:36:38.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:38.811+0000 7f75364b5700 1 -- 172.21.15.110:0/2187564511 learned_addr learned my addr 172.21.15.110:0/2187564511 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:36:38.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:38.811+0000 7f7535cb4700 1 --2- 172.21.15.110:0/2187564511 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7530068680 0x7f753013a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:38.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:38.811+0000 7f75354b3700 1 --2- 172.21.15.110:0/2187564511 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f75300ac5c0 0x7f753013d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:38.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:38.811+0000 7f75364b5700 1 -- 172.21.15.110:0/2187564511 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f75300ac5c0 msgr2=0x7f753013d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:38.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:38.811+0000 7f75364b5700 1 --2- 172.21.15.110:0/2187564511 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f75300ac5c0 0x7f753013d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:38.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:38.811+0000 7f75364b5700 1 -- 172.21.15.110:0/2187564511 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7530068680 msgr2=0x7f753013a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:38.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:38.811+0000 7f75364b5700 1 --2- 172.21.15.110:0/2187564511 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7530068680 0x7f753013a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:38.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:38.811+0000 7f75364b5700 1 -- 172.21.15.110:0/2187564511 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7528009ae0 con 0x7f7530137a70 2024-09-15T03:36:38.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:38.811+0000 7f75364b5700 1 --2- 172.21.15.110:0/2187564511 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f7530137a70 0x7f753014e9a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f752000ea80 tx=0x7f75200091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:38.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:38.815+0000 7f7525ffb700 1 -- 172.21.15.110:0/2187564511 <== mon.2 v2:172.21.15.110:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752000c7f0 con 0x7f7530137a70 2024-09-15T03:36:38.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:38.815+0000 7f7525ffb700 1 -- 172.21.15.110:0/2187564511 <== mon.2 v2:172.21.15.110:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f752000c970 con 0x7f7530137a70 2024-09-15T03:36:38.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:38.815+0000 7f7536cb6700 1 -- 172.21.15.110:0/2187564511 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75300a44d0 con 0x7f7530137a70 2024-09-15T03:36:38.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:38.815+0000 7f7525ffb700 1 -- 172.21.15.110:0/2187564511 <== mon.2 v2:172.21.15.110:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752000cd30 con 0x7f7530137a70 2024-09-15T03:36:38.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:38.815+0000 7f7536cb6700 1 -- 172.21.15.110:0/2187564511 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f753014f100 con 0x7f7530137a70 2024-09-15T03:36:38.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:38.815+0000 7f7536cb6700 1 -- 172.21.15.110:0/2187564511 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75300716d0 con 0x7f7530137a70 2024-09-15T03:36:38.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:38.815+0000 7f7525ffb700 1 -- 172.21.15.110:0/2187564511 <== mon.2 v2:172.21.15.110:3301/0 4 ==== mgrmap(e 4) v1 ==== 85791+0+0 (secure 0 0 0) 0x7f7520013680 con 0x7f7530137a70 2024-09-15T03:36:38.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:38.815+0000 7f7525ffb700 1 --2- 172.21.15.110:0/2187564511 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f75140679f0 0x7f7514069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:38.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:38.815+0000 7f7535cb4700 1 --2- 172.21.15.110:0/2187564511 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f75140679f0 0x7f7514069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:38.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:38.815+0000 7f7525ffb700 1 -- 172.21.15.110:0/2187564511 <== mon.2 v2:172.21.15.110:3301/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f7520085550 con 0x7f7530137a70 2024-09-15T03:36:38.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:38.819+0000 7f7535cb4700 1 --2- 172.21.15.110:0/2187564511 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f75140679f0 0x7f7514069ea0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7528009880 tx=0x7f7528018000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:38.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:38.823+0000 7f7525ffb700 1 -- 172.21.15.110:0/2187564511 <== mon.2 v2:172.21.15.110:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f7520052a80 con 0x7f7530137a70 2024-09-15T03:36:39.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.011+0000 7f7536cb6700 1 -- 172.21.15.110:0/2187564511 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f7530068d40 con 0x7f7530137a70 2024-09-15T03:36:39.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.651+0000 7f7525ffb700 1 -- 172.21.15.110:0/2187564511 <== mon.2 v2:172.21.15.110:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v16) v1 ==== 109+0+0 (secure 0 0 0) 0x7f7520055b00 con 0x7f7530137a70 2024-09-15T03:36:39.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.655+0000 7f7536cb6700 1 -- 172.21.15.110:0/2187564511 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f75140679f0 msgr2=0x7f7514069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:39.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.655+0000 7f7536cb6700 1 --2- 172.21.15.110:0/2187564511 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f75140679f0 0x7f7514069ea0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7528009880 tx=0x7f7528018000 comp rx=0 tx=0).stop 2024-09-15T03:36:39.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.655+0000 7f7536cb6700 1 -- 172.21.15.110:0/2187564511 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f7530137a70 msgr2=0x7f753014e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:39.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.655+0000 7f7536cb6700 1 --2- 172.21.15.110:0/2187564511 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f7530137a70 0x7f753014e9a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f752000ea80 tx=0x7f75200091a0 comp rx=0 tx=0).stop 2024-09-15T03:36:39.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.655+0000 7f7536cb6700 1 -- 172.21.15.110:0/2187564511 shutdown_connections 2024-09-15T03:36:39.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.655+0000 7f7536cb6700 1 --2- 172.21.15.110:0/2187564511 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f75140679f0 0x7f7514069ea0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:39.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.655+0000 7f7536cb6700 1 --2- 172.21.15.110:0/2187564511 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f7530137a70 0x7f753014e9a0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:39.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.655+0000 7f7536cb6700 1 --2- 172.21.15.110:0/2187564511 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f75300ac5c0 0x7f753013d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:39.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.655+0000 7f7536cb6700 1 --2- 172.21.15.110:0/2187564511 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7530068680 0x7f753013a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:39.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.655+0000 7f7536cb6700 1 -- 172.21.15.110:0/2187564511 >> 172.21.15.110:0/2187564511 conn(0x7f75300b1040 msgr2=0x7f7530132db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:39.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.655+0000 7f7536cb6700 1 -- 172.21.15.110:0/2187564511 shutdown_connections 2024-09-15T03:36:39.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.655+0000 7f7536cb6700 1 -- 172.21.15.110:0/2187564511 wait complete. 2024-09-15T03:36:39.661 INFO:teuthology.orchestra.run.smithi110.stderr:pool 'rbd' created 2024-09-15T03:36:39.672 DEBUG:teuthology.orchestra.run.smithi110:> rbd --cluster ceph pool init rbd 2024-09-15T03:36:39.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.711+0000 7f1bc8fd94c0 1 Processor -- start 2024-09-15T03:36:39.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.711+0000 7f1bc8fd94c0 1 -- start start 2024-09-15T03:36:39.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.711+0000 7f1bc8fd94c0 1 --2- >> v2:172.21.15.110:3300/0 conn(0x55eae68cadd0 0x55eae68bed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:39.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.711+0000 7f1bc8fd94c0 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x55eae67d4660 con 0x55eae68a0a80 2024-09-15T03:36:39.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.711+0000 7f1bc8fd94c0 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x55eae67ad7f0 con 0x55eae68c6be0 2024-09-15T03:36:39.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.711+0000 7f1bc8fd94c0 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 v1 -- 0x55eae680b360 con 0x55eae68cadd0 2024-09-15T03:36:39.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.711+0000 7f1bc730e700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x55eae68cadd0 0x55eae68bed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:39.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.711+0000 7f1bbffff700 1 --1- >> v1:172.21.15.110:6789/0 conn(0x55eae68a0a80 0x55eae6957100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:39896/0 (socket says 172.21.15.110:39896) 2024-09-15T03:36:39.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.711+0000 7f1bbffff700 1 -- 172.21.15.110:0/1175399258 learned_addr learned my addr 172.21.15.110:0/1175399258 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-15T03:36:39.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.711+0000 7f1bc730e700 1 -- 172.21.15.110:0/1175399258 >> v1:172.21.15.177:6789/0 conn(0x55eae68c6be0 legacy=0x55eae68c7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:39.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.711+0000 7f1bc730e700 1 -- 172.21.15.110:0/1175399258 >> v1:172.21.15.110:6789/0 conn(0x55eae68a0a80 legacy=0x55eae6957100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:39.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.711+0000 7f1bc730e700 1 -- 172.21.15.110:0/1175399258 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eae67ba660 con 0x55eae68cadd0 2024-09-15T03:36:39.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.711+0000 7f1bc730e700 1 --2- 172.21.15.110:0/1175399258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eae68cadd0 0x55eae68bed90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1bb800a650 tx=0x7f1bb8005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d2b010f6ffba9bd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:39.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.711+0000 7f1bc6b0d700 1 -- 172.21.15.110:0/1175399258 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb8008700 con 0x55eae68cadd0 2024-09-15T03:36:39.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.711+0000 7f1bc6b0d700 1 -- 172.21.15.110:0/1175399258 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bb8008860 con 0x55eae68cadd0 2024-09-15T03:36:39.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.711+0000 7f1bc6b0d700 1 -- 172.21.15.110:0/1175399258 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb8008c00 con 0x55eae68cadd0 2024-09-15T03:36:39.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.715+0000 7f1bc8fd94c0 1 -- 172.21.15.110:0/1175399258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eae68cadd0 msgr2=0x55eae68bed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:39.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.715+0000 7f1bc8fd94c0 1 --2- 172.21.15.110:0/1175399258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eae68cadd0 0x55eae68bed90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1bb800a650 tx=0x7f1bb8005a50 comp rx=0 tx=0).stop 2024-09-15T03:36:39.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.715+0000 7f1bc8fd94c0 1 -- 172.21.15.110:0/1175399258 shutdown_connections 2024-09-15T03:36:39.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.715+0000 7f1bc8fd94c0 1 --2- 172.21.15.110:0/1175399258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eae68cadd0 0x55eae68bed90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:39.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.715+0000 7f1bc8fd94c0 1 -- 172.21.15.110:0/1175399258 >> 172.21.15.110:0/1175399258 conn(0x55eae68cddf0 msgr2=0x55eae68ce1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:39.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.715+0000 7f1bc8fd94c0 1 -- 172.21.15.110:0/1175399258 shutdown_connections 2024-09-15T03:36:39.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.715+0000 7f1bc8fd94c0 1 -- 172.21.15.110:0/1175399258 wait complete. 2024-09-15T03:36:39.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.715+0000 7f1bc8fd94c0 1 Processor -- start 2024-09-15T03:36:39.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.715+0000 7f1bc8fd94c0 1 -- start start 2024-09-15T03:36:39.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.719+0000 7f1bc8fd94c0 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eae68a0a80 0x55eae6958080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:39.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.719+0000 7f1bc8fd94c0 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x55eae68c6be0 0x55eae694ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:39.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.719+0000 7f1bc8fd94c0 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eae68cadd0 0x55eae694f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:39.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.719+0000 7f1bc8fd94c0 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eae679fbc0 con 0x55eae68cadd0 2024-09-15T03:36:39.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.719+0000 7f1bc8fd94c0 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eae67cbd10 con 0x55eae68a0a80 2024-09-15T03:36:39.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.719+0000 7f1bc8fd94c0 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eae67aef40 con 0x55eae68c6be0 2024-09-15T03:36:39.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.719+0000 7f1bbffff700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x55eae68c6be0 0x55eae694ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:39.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.719+0000 7f1bc7b0f700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eae68cadd0 0x55eae694f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:39.723 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.719+0000 7f1bbffff700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x55eae68c6be0 0x55eae694ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3301/0 says I am v2:172.21.15.110:44620/0 (socket says 172.21.15.110:44620) 2024-09-15T03:36:39.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.719+0000 7f1bbffff700 1 -- 172.21.15.110:0/3730682564 learned_addr learned my addr 172.21.15.110:0/3730682564 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:36:39.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.719+0000 7f1bc730e700 1 --2- 172.21.15.110:0/3730682564 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eae68a0a80 0x55eae6958080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:39.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.719+0000 7f1bbffff700 1 -- 172.21.15.110:0/3730682564 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eae68a0a80 msgr2=0x55eae6958080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:39.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.719+0000 7f1bbffff700 1 --2- 172.21.15.110:0/3730682564 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eae68a0a80 0x55eae6958080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:39.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.719+0000 7f1bbffff700 1 -- 172.21.15.110:0/3730682564 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eae68cadd0 msgr2=0x55eae694f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:39.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.719+0000 7f1bbffff700 1 --2- 172.21.15.110:0/3730682564 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eae68cadd0 0x55eae694f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:39.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.719+0000 7f1bbffff700 1 -- 172.21.15.110:0/3730682564 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb8005cf0 con 0x55eae68c6be0 2024-09-15T03:36:39.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.719+0000 7f1bbffff700 1 --2- 172.21.15.110:0/3730682564 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x55eae68c6be0 0x55eae694ec10 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1bb000bdb0 tx=0x7f1bb00017e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:39.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.719+0000 7f1bbf7fe700 1 -- 172.21.15.110:0/3730682564 <== mon.2 v2:172.21.15.110:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb0009c70 con 0x55eae68c6be0 2024-09-15T03:36:39.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.719+0000 7f1bbf7fe700 1 -- 172.21.15.110:0/3730682564 <== mon.2 v2:172.21.15.110:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bb0009dd0 con 0x55eae68c6be0 2024-09-15T03:36:39.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.719+0000 7f1bc8fd94c0 1 -- 172.21.15.110:0/3730682564 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55eae67adb80 con 0x55eae68c6be0 2024-09-15T03:36:39.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.719+0000 7f1bbf7fe700 1 -- 172.21.15.110:0/3730682564 <== mon.2 v2:172.21.15.110:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb000e6c0 con 0x55eae68c6be0 2024-09-15T03:36:39.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.719+0000 7f1bc8fd94c0 1 -- 172.21.15.110:0/3730682564 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eae67ad5e0 con 0x55eae68c6be0 2024-09-15T03:36:39.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.723+0000 7f1bbf7fe700 1 -- 172.21.15.110:0/3730682564 <== mon.2 v2:172.21.15.110:3301/0 4 ==== mgrmap(e 4) v1 ==== 85791+0+0 (secure 0 0 0) 0x7f1bb000eab0 con 0x55eae68c6be0 2024-09-15T03:36:39.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.723+0000 7f1bbf7fe700 1 --2- 172.21.15.110:0/3730682564 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f1ba8066290 0x7f1ba8068740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:39.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.723+0000 7f1bbf7fe700 1 -- 172.21.15.110:0/3730682564 <== mon.2 v2:172.21.15.110:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5195+0+0 (secure 0 0 0) 0x7f1bb0084330 con 0x55eae68c6be0 2024-09-15T03:36:39.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.723+0000 7f1bc730e700 1 --2- 172.21.15.110:0/3730682564 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f1ba8066290 0x7f1ba8068740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:39.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.723+0000 7f1bc8fd94c0 1 -- 172.21.15.110:0/3730682564 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_command({"prefix": "osd pool application enable","pool": "rbd","app": "rbd"} v 0) v1 -- 0x55eae694f690 con 0x55eae68c6be0 2024-09-15T03:36:39.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:39.723+0000 7f1bc730e700 1 --2- 172.21.15.110:0/3730682564 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f1ba8066290 0x7f1ba8068740 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1bb8005a90 tx=0x7f1bb8016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:40.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:40.663+0000 7f1bbf7fe700 1 -- 172.21.15.110:0/3730682564 <== mon.2 v2:172.21.15.110:3301/0 6 ==== mon_command_ack([{"prefix": "osd pool application enable","pool": "rbd","app": "rbd"}]=0 enabled application 'rbd' on pool 'rbd' v17) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1bb001c020 con 0x55eae68c6be0 2024-09-15T03:36:40.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:40.663+0000 7f1bc8fd94c0 1 -- 172.21.15.110:0/3730682564 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x55eae67aed30 con 0x55eae68c6be0 2024-09-15T03:36:40.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:40.663+0000 7f1bbf7fe700 1 -- 172.21.15.110:0/3730682564 <== mon.2 v2:172.21.15.110:3301/0 7 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1bb0082bf0 con 0x55eae68c6be0 2024-09-15T03:36:40.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:40.663+0000 7f1bc630c700 1 -- 172.21.15.110:0/3730682564 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({osdmap=17}) v3 -- 0x7f1bac002990 con 0x55eae68c6be0 2024-09-15T03:36:40.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:40.663+0000 7f1bbf7fe700 1 -- 172.21.15.110:0/3730682564 <== mon.2 v2:172.21.15.110:3301/0 8 ==== osd_map(17..17 src has 1..17) v4 ==== 658+0+0 (secure 0 0 0) 0x7f1bb0054b30 con 0x55eae68c6be0 2024-09-15T03:36:40.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:40.663+0000 7f1bc8fd94c0 1 --2- 172.21.15.110:0/3730682564 >> [v2:172.21.15.177:6802/22424,v1:172.21.15.177:6804/22424] conn(0x55eae6969050 0x55eae6969420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:40.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:40.663+0000 7f1bc8fd94c0 1 -- 172.21.15.110:0/3730682564 --> [v2:172.21.15.177:6802/22424,v1:172.21.15.177:6804/22424] -- osd_op(unknown.0.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17) v8 -- 0x55eae69667b0 con 0x55eae6969050 2024-09-15T03:36:40.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:40.663+0000 7f1bc7b0f700 1 --2- 172.21.15.110:0/3730682564 >> [v2:172.21.15.177:6802/22424,v1:172.21.15.177:6804/22424] conn(0x55eae6969050 0x55eae6969420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:40.670 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:40.663+0000 7f1bc7b0f700 1 --2- 172.21.15.110:0/3730682564 >> [v2:172.21.15.177:6802/22424,v1:172.21.15.177:6804/22424] conn(0x55eae6969050 0x55eae6969420 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:40.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:40.671+0000 7f1bc7b0f700 1 -- 172.21.15.110:0/3730682564 <== osd.5 v2:172.21.15.177:6802/22424 1 ==== osd_op_reply(1 rbd_trash [create] v17'1 uv1 ondisk = 0) v8 ==== 153+0+0 (crc 0 0 0) 0x7f1bc00026e0 con 0x55eae6969050 2024-09-15T03:36:40.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:40.675+0000 7f1bc8fd94c0 1 -- 172.21.15.110:0/3730682564 --> [v2:172.21.15.177:6802/22424,v1:172.21.15.177:6804/22424] -- osd_op(unknown.0.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=33b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e17) v8 -- 0x55eae69bedd0 con 0x55eae6969050 2024-09-15T03:36:40.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:40.675+0000 7f1bc7b0f700 1 -- 172.21.15.110:0/3730682564 <== osd.5 v2:172.21.15.177:6802/22424 2 ==== osd_op_reply(2 rbd_info [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1bc00026e0 con 0x55eae6969050 2024-09-15T03:36:40.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:40.675+0000 7f1bc8fd94c0 1 -- 172.21.15.110:0/3730682564 --> [v2:172.21.15.177:6802/22424,v1:172.21.15.177:6804/22424] -- osd_op(unknown.0.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e17) v8 -- 0x55eae69bdee0 con 0x55eae6969050 2024-09-15T03:36:40.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:40.675+0000 7f1bc7b0f700 1 -- 172.21.15.110:0/3730682564 <== osd.5 v2:172.21.15.177:6802/22424 3 ==== osd_op_reply(3 rbd_info [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1bc00026e0 con 0x55eae6969050 2024-09-15T03:36:40.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:40.675+0000 7f1bc5b0b700 1 -- 172.21.15.110:0/3730682564 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- pool_op(create unmanaged snap pool 2 tid 4 name v0) v4 -- 0x7f1ba0001ee0 con 0x55eae68c6be0 2024-09-15T03:36:41.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:41.663+0000 7f1bbf7fe700 1 -- 172.21.15.110:0/3730682564 <== mon.2 v2:172.21.15.110:3301/0 9 ==== mgrmap(e 5) v1 ==== 85847+0+0 (secure 0 0 0) 0x7f1bb0084030 con 0x55eae68c6be0 2024-09-15T03:36:41.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:41.671+0000 7f1bbf7fe700 1 -- 172.21.15.110:0/3730682564 <== mon.2 v2:172.21.15.110:3301/0 10 ==== pool_op_reply(tid 4 (0) Success v18) v1 ==== 55+0+0 (secure 0 0 0) 0x7f1bb0054790 con 0x55eae68c6be0 2024-09-15T03:36:41.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:41.671+0000 7f1bbf7fe700 1 -- 172.21.15.110:0/3730682564 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({osdmap=18}) v3 -- 0x7f1ba8027460 con 0x55eae68c6be0 2024-09-15T03:36:41.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:41.671+0000 7f1bbf7fe700 1 -- 172.21.15.110:0/3730682564 <== mon.2 v2:172.21.15.110:3301/0 11 ==== osd_map(18..18 src has 1..18) v4 ==== 618+0+0 (secure 0 0 0) 0x7f1bb0076370 con 0x55eae68c6be0 2024-09-15T03:36:41.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:41.671+0000 7f1bc630c700 1 -- 172.21.15.110:0/3730682564 --> [v2:172.21.15.177:6802/22424,v1:172.21.15.177:6804/22424] -- osd_op(unknown.0.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8 in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18) v8 -- 0x7f1bac005450 con 0x55eae6969050 2024-09-15T03:36:41.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:41.679+0000 7f1bc7b0f700 1 -- 172.21.15.110:0/3730682564 <== osd.5 v2:172.21.15.177:6802/22424 4 ==== osd_op_reply(5 rbd_info [create,write 0~8] v18'1 uv1 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x7f1bc00026e0 con 0x55eae6969050 2024-09-15T03:36:41.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:41.679+0000 7f1bc5b0b700 1 -- 172.21.15.110:0/3730682564 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- pool_op(delete unmanaged snap pool 2 tid 6 name v18) v4 -- 0x7f1ba0003360 con 0x55eae68c6be0 2024-09-15T03:36:42.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.683+0000 7f1bbf7fe700 1 -- 172.21.15.110:0/3730682564 <== mon.2 v2:172.21.15.110:3301/0 12 ==== pool_op_reply(tid 6 (0) Success v19) v1 ==== 43+0+0 (secure 0 0 0) 0x7f1bb0073020 con 0x55eae68c6be0 2024-09-15T03:36:42.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.683+0000 7f1bbf7fe700 1 -- 172.21.15.110:0/3730682564 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({osdmap=19}) v3 -- 0x7f1ba8027460 con 0x55eae68c6be0 2024-09-15T03:36:42.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.683+0000 7f1bbf7fe700 1 -- 172.21.15.110:0/3730682564 <== mon.2 v2:172.21.15.110:3301/0 13 ==== osd_map(19..19 src has 1..19) v4 ==== 646+0+0 (secure 0 0 0) 0x7f1bb0017400 con 0x55eae68c6be0 2024-09-15T03:36:42.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.683+0000 7f1bc630c700 1 -- 172.21.15.110:0/3730682564 --> [v2:172.21.15.177:6802/22424,v1:172.21.15.177:6804/22424] -- osd_op(unknown.0.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19 in=19b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19) v8 -- 0x7f1bac005820 con 0x55eae6969050 2024-09-15T03:36:42.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.691+0000 7f1bc7b0f700 1 -- 172.21.15.110:0/3730682564 <== osd.5 v2:172.21.15.177:6802/22424 5 ==== osd_op_reply(7 rbd_info [write 0~19] v19'2 uv2 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1bc00026e0 con 0x55eae6969050 2024-09-15T03:36:42.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.691+0000 7f1bc8fd94c0 1 -- 172.21.15.110:0/3730682564 >> [v2:172.21.15.177:6802/22424,v1:172.21.15.177:6804/22424] conn(0x55eae6969050 msgr2=0x55eae6969420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:42.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.691+0000 7f1bc8fd94c0 1 --2- 172.21.15.110:0/3730682564 >> [v2:172.21.15.177:6802/22424,v1:172.21.15.177:6804/22424] conn(0x55eae6969050 0x55eae6969420 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:42.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.691+0000 7f1bc8fd94c0 1 -- 172.21.15.110:0/3730682564 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f1ba8066290 msgr2=0x7f1ba8068740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:42.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.691+0000 7f1bc8fd94c0 1 --2- 172.21.15.110:0/3730682564 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f1ba8066290 0x7f1ba8068740 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1bb8005a90 tx=0x7f1bb8016000 comp rx=0 tx=0).stop 2024-09-15T03:36:42.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.691+0000 7f1bc8fd94c0 1 -- 172.21.15.110:0/3730682564 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x55eae68c6be0 msgr2=0x55eae694ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:42.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.691+0000 7f1bc8fd94c0 1 --2- 172.21.15.110:0/3730682564 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x55eae68c6be0 0x55eae694ec10 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1bb000bdb0 tx=0x7f1bb00017e0 comp rx=0 tx=0).stop 2024-09-15T03:36:42.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.691+0000 7f1bc7b0f700 1 -- 172.21.15.110:0/3730682564 reap_dead start 2024-09-15T03:36:42.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.691+0000 7f1bc8fd94c0 1 -- 172.21.15.110:0/3730682564 shutdown_connections 2024-09-15T03:36:42.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.691+0000 7f1bc8fd94c0 1 -- 172.21.15.110:0/3730682564 >> 172.21.15.110:0/3730682564 conn(0x55eae68cddf0 msgr2=0x55eae6899550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:42.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.691+0000 7f1bc8fd94c0 1 -- 172.21.15.110:0/3730682564 shutdown_connections 2024-09-15T03:36:42.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.691+0000 7f1bc8fd94c0 1 -- 172.21.15.110:0/3730682564 wait complete. 2024-09-15T03:36:42.701 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-15T03:36:42.701 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-15T03:36:42.701 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-15T03:36:42.702 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-15T03:36:42.702 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-15T03:36:42.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.807+0000 7f209e23f700 1 Processor -- start 2024-09-15T03:36:42.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.807+0000 7f209e23f700 1 -- start start 2024-09-15T03:36:42.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.811+0000 7f209e23f700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f20980a4d70 0x7f2098140d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:42.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.811+0000 7f209e23f700 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20980a6850 con 0x7f20980aef20 2024-09-15T03:36:42.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.811+0000 7f209e23f700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20981415c0 con 0x7f20980aeae0 2024-09-15T03:36:42.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.811+0000 7f209e23f700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20980a5140 con 0x7f20980a4d70 2024-09-15T03:36:42.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.811+0000 7f2097fff700 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f20980aef20 0x7f209813df40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:39900/0 (socket says 172.21.15.110:39900) 2024-09-15T03:36:42.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.811+0000 7f2097fff700 1 -- 172.21.15.110:0/456148695 learned_addr learned my addr 172.21.15.110:0/456148695 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-15T03:36:42.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.811+0000 7f20977fe700 1 --2- 172.21.15.110:0/456148695 >> v2:172.21.15.177:3300/0 conn(0x7f20980a4d70 0x7f2098140d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:42.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.811+0000 7f20967fc700 1 -- 172.21.15.110:0/456148695 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3579720797 0 0) 0x7f20980a6850 con 0x7f20980aef20 2024-09-15T03:36:42.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.811+0000 7f20967fc700 1 -- 172.21.15.110:0/456148695 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2084003570 con 0x7f20980aef20 2024-09-15T03:36:42.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.811+0000 7f20977fe700 1 -- 172.21.15.110:0/456148695 >> v1:172.21.15.177:6789/0 conn(0x7f20980aeae0 legacy=0x7f209813d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:42.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.811+0000 7f20977fe700 1 -- 172.21.15.110:0/456148695 >> v1:172.21.15.110:6789/0 conn(0x7f20980aef20 legacy=0x7f209813df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:42.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.811+0000 7f20977fe700 1 -- 172.21.15.110:0/456148695 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f209813e650 con 0x7f20980a4d70 2024-09-15T03:36:42.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.811+0000 7f20977fe700 1 --2- 172.21.15.110:0/456148695 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f20980a4d70 0x7f2098140d40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2088009e60 tx=0x7f2088009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13e7f3d2f2d1ca10 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:42.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.811+0000 7f20967fc700 1 -- 172.21.15.110:0/456148695 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208800e070 con 0x7f20980a4d70 2024-09-15T03:36:42.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.811+0000 7f20967fc700 1 -- 172.21.15.110:0/456148695 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f208800bb60 con 0x7f20980a4d70 2024-09-15T03:36:42.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.811+0000 7f20967fc700 1 -- 172.21.15.110:0/456148695 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2088012430 con 0x7f20980a4d70 2024-09-15T03:36:42.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.811+0000 7f209e23f700 1 -- 172.21.15.110:0/456148695 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f20980a4d70 msgr2=0x7f2098140d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:42.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.811+0000 7f209e23f700 1 --2- 172.21.15.110:0/456148695 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f20980a4d70 0x7f2098140d40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2088009e60 tx=0x7f2088009780 comp rx=0 tx=0).stop 2024-09-15T03:36:42.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.811+0000 7f209e23f700 1 -- 172.21.15.110:0/456148695 shutdown_connections 2024-09-15T03:36:42.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.811+0000 7f209e23f700 1 --2- 172.21.15.110:0/456148695 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f20980a4d70 0x7f2098140d40 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:42.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.811+0000 7f209e23f700 1 -- 172.21.15.110:0/456148695 >> 172.21.15.110:0/456148695 conn(0x7f209806f870 msgr2=0x7f20980b4170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:42.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.811+0000 7f209e23f700 1 -- 172.21.15.110:0/456148695 shutdown_connections 2024-09-15T03:36:42.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.811+0000 7f209e23f700 1 -- 172.21.15.110:0/456148695 wait complete. 2024-09-15T03:36:42.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.811+0000 7f209e23f700 1 Processor -- start 2024-09-15T03:36:42.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.811+0000 7f209e23f700 1 -- start start 2024-09-15T03:36:42.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.815+0000 7f209e23f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f20980a4d70 0x7f20981409e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:42.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.815+0000 7f209e23f700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f20980aeae0 0x7f2098150b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:42.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.815+0000 7f209e23f700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f20980aef20 0x7f2098151090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:42.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.815+0000 7f209e23f700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20981515d0 con 0x7f20980aeae0 2024-09-15T03:36:42.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.815+0000 7f209e23f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20980a8ea0 con 0x7f20980a4d70 2024-09-15T03:36:42.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.815+0000 7f209e23f700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20980a9330 con 0x7f20980aef20 2024-09-15T03:36:42.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.815+0000 7f2096ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f20980aeae0 0x7f2098150b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:42.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.815+0000 7f2097fff700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f20980aef20 0x7f2098151090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:42.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.815+0000 7f2096ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f20980aeae0 0x7f2098150b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37192/0 (socket says 172.21.15.110:37192) 2024-09-15T03:36:42.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.815+0000 7f2096ffd700 1 -- 172.21.15.110:0/1881823453 learned_addr learned my addr 172.21.15.110:0/1881823453 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:36:42.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.815+0000 7f20977fe700 1 --2- 172.21.15.110:0/1881823453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f20980a4d70 0x7f20981409e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:42.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.815+0000 7f2096ffd700 1 -- 172.21.15.110:0/1881823453 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f20980aef20 msgr2=0x7f2098151090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:42.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.815+0000 7f2096ffd700 1 --2- 172.21.15.110:0/1881823453 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f20980aef20 0x7f2098151090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:42.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.815+0000 7f2096ffd700 1 -- 172.21.15.110:0/1881823453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f20980a4d70 msgr2=0x7f20981409e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:42.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.815+0000 7f2096ffd700 1 --2- 172.21.15.110:0/1881823453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f20980a4d70 0x7f20981409e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:42.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.815+0000 7f2096ffd700 1 -- 172.21.15.110:0/1881823453 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2088009a20 con 0x7f20980aeae0 2024-09-15T03:36:42.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.815+0000 7f2097fff700 1 --2- 172.21.15.110:0/1881823453 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f20980aef20 0x7f2098151090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T03:36:42.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.815+0000 7f20977fe700 1 --2- 172.21.15.110:0/1881823453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f20980a4d70 0x7f20981409e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T03:36:42.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.815+0000 7f2096ffd700 1 --2- 172.21.15.110:0/1881823453 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f20980aeae0 0x7f2098150b50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f208000c590 tx=0x7f208000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:42.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.815+0000 7f208f7fe700 1 -- 172.21.15.110:0/1881823453 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2080011590 con 0x7f20980aeae0 2024-09-15T03:36:42.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.815+0000 7f208f7fe700 1 -- 172.21.15.110:0/1881823453 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2080011710 con 0x7f20980aeae0 2024-09-15T03:36:42.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.815+0000 7f209e23f700 1 -- 172.21.15.110:0/1881823453 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20980a9550 con 0x7f20980aeae0 2024-09-15T03:36:42.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.815+0000 7f208f7fe700 1 -- 172.21.15.110:0/1881823453 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2080011ad0 con 0x7f20980aeae0 2024-09-15T03:36:42.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.815+0000 7f209e23f700 1 -- 172.21.15.110:0/1881823453 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20980a9af0 con 0x7f20980aeae0 2024-09-15T03:36:42.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.819+0000 7f209e23f700 1 -- 172.21.15.110:0/1881823453 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20980716d0 con 0x7f20980aeae0 2024-09-15T03:36:42.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.819+0000 7f208f7fe700 1 -- 172.21.15.110:0/1881823453 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) v1 ==== 85847+0+0 (secure 0 0 0) 0x7f208001b3b0 con 0x7f20980aeae0 2024-09-15T03:36:42.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.819+0000 7f208f7fe700 1 --2- 172.21.15.110:0/1881823453 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f2078067a80 0x7f2078069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:42.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.819+0000 7f208f7fe700 1 -- 172.21.15.110:0/1881823453 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5262+0+0 (secure 0 0 0) 0x7f20800849d0 con 0x7f20980aeae0 2024-09-15T03:36:42.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.819+0000 7f20977fe700 1 --2- 172.21.15.110:0/1881823453 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f2078067a80 0x7f2078069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:42.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.819+0000 7f20977fe700 1 --2- 172.21.15.110:0/1881823453 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f2078067a80 0x7f2078069f30 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f20880097c0 tx=0x7f2088019000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:42.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:42.827+0000 7f208f7fe700 1 -- 172.21.15.110:0/1881823453 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f2080051c90 con 0x7f20980aeae0 2024-09-15T03:36:43.033 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.027+0000 7f209e23f700 1 -- 172.21.15.110:0/1881823453 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f20980a64c0 con 0x7f20980aeae0 2024-09-15T03:36:43.041 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.035+0000 7f208f7fe700 1 -- 172.21.15.110:0/1881823453 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+166496 (secure 0 0 0) 0x7f2080017070 con 0x7f20980aeae0 2024-09-15T03:36:43.044 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-15T03:36:43.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.039+0000 7f209e23f700 1 -- 172.21.15.110:0/1881823453 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f2078067a80 msgr2=0x7f2078069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.039+0000 7f209e23f700 1 --2- 172.21.15.110:0/1881823453 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f2078067a80 0x7f2078069f30 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f20880097c0 tx=0x7f2088019000 comp rx=0 tx=0).stop 2024-09-15T03:36:43.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.039+0000 7f209e23f700 1 -- 172.21.15.110:0/1881823453 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f20980aeae0 msgr2=0x7f2098150b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.039+0000 7f209e23f700 1 --2- 172.21.15.110:0/1881823453 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f20980aeae0 0x7f2098150b50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f208000c590 tx=0x7f208000c8a0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.039+0000 7f209e23f700 1 -- 172.21.15.110:0/1881823453 shutdown_connections 2024-09-15T03:36:43.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.039+0000 7f209e23f700 1 --2- 172.21.15.110:0/1881823453 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f2078067a80 0x7f2078069f30 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.039+0000 7f209e23f700 1 --2- 172.21.15.110:0/1881823453 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f20980aef20 0x7f2098151090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.039+0000 7f209e23f700 1 --2- 172.21.15.110:0/1881823453 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f20980aeae0 0x7f2098150b50 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.039+0000 7f209e23f700 1 --2- 172.21.15.110:0/1881823453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f20980a4d70 0x7f20981409e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.039+0000 7f209e23f700 1 -- 172.21.15.110:0/1881823453 >> 172.21.15.110:0/1881823453 conn(0x7f209806f870 msgr2=0x7f20980b14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:43.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.039+0000 7f209e23f700 1 -- 172.21.15.110:0/1881823453 shutdown_connections 2024-09-15T03:36:43.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.039+0000 7f209e23f700 1 -- 172.21.15.110:0/1881823453 wait complete. 2024-09-15T03:36:43.058 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":5,"flags":0,"active_gid":4104,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":22190},{"type":"v1","addr":"172.21.15.110:6833","nonce":22190}]},"active_addr":"172.21.15.110:6833/22190","active_change":"2024-09-15T03:36:32.590610+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":4123,"name":"x","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading asyncssh library:No module named 'asyncssh'","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading asyncssh library:No module named 'asyncssh'","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.110:0","nonce":2273796391}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.110:0","nonce":1132811516}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.110:0","nonce":3472177498}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.110:0","nonce":1467495725}]}]} 2024-09-15T03:36:43.061 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-15T03:36:43.061 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-15T03:36:43.061 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T03:36:43.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.159+0000 7f0707fff700 1 Processor -- start 2024-09-15T03:36:43.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.159+0000 7f0707fff700 1 -- start start 2024-09-15T03:36:43.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.159+0000 7f0707fff700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f07080ae9e0 0x7f07080aee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:43.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.159+0000 7f0707fff700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f070813a380 0x7f070813a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:43.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.159+0000 7f0707fff700 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07081414f0 con 0x7f07080a4d70 2024-09-15T03:36:43.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.159+0000 7f0707fff700 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07080a67a0 con 0x7f070813a380 2024-09-15T03:36:43.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.159+0000 7f0707fff700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07080a5140 con 0x7f07080ae9e0 2024-09-15T03:36:43.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.159+0000 7f07077fe700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f070813a380 0x7f070813a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:43.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.159+0000 7f07077fe700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f070813a380 0x7f070813a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37196/0 (socket says 172.21.15.110:37196) 2024-09-15T03:36:43.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.159+0000 7f07077fe700 1 -- 172.21.15.110:0/3744123566 learned_addr learned my addr 172.21.15.110:0/3744123566 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:36:43.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.159+0000 7f06fe7fc700 1 --2- 172.21.15.110:0/3744123566 >> v2:172.21.15.177:3300/0 conn(0x7f07080ae9e0 0x7f07080aee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:43.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.159+0000 7f07077fe700 1 -- 172.21.15.110:0/3744123566 >> v2:172.21.15.177:3300/0 conn(0x7f07080ae9e0 msgr2=0x7f07080aee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:36:43.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.159+0000 7f07077fe700 1 --2- 172.21.15.110:0/3744123566 >> v2:172.21.15.177:3300/0 conn(0x7f07080ae9e0 0x7f07080aee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.159+0000 7f07077fe700 1 -- 172.21.15.110:0/3744123566 >> v1:172.21.15.110:6789/0 conn(0x7f07080a4d70 legacy=0x7f0708140c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.159+0000 7f07077fe700 1 -- 172.21.15.110:0/3744123566 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07080af380 con 0x7f070813a380 2024-09-15T03:36:43.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.159+0000 7f07077fe700 1 --2- 172.21.15.110:0/3744123566 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f070813a380 0x7f070813a7a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0700009f20 tx=0x7f0700009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be9597fba406b387 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:43.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.159+0000 7f07067fc700 1 -- 172.21.15.110:0/3744123566 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f070000e070 con 0x7f070813a380 2024-09-15T03:36:43.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.159+0000 7f07067fc700 1 -- 172.21.15.110:0/3744123566 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f070000b990 con 0x7f070813a380 2024-09-15T03:36:43.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.159+0000 7f07067fc700 1 -- 172.21.15.110:0/3744123566 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07000092e0 con 0x7f070813a380 2024-09-15T03:36:43.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.163+0000 7f0707fff700 1 -- 172.21.15.110:0/3744123566 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f070813a380 msgr2=0x7f070813a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.163+0000 7f0707fff700 1 --2- 172.21.15.110:0/3744123566 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f070813a380 0x7f070813a7a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0700009f20 tx=0x7f0700009840 comp rx=0 tx=0).stop 2024-09-15T03:36:43.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.163+0000 7f0707fff700 1 -- 172.21.15.110:0/3744123566 shutdown_connections 2024-09-15T03:36:43.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.163+0000 7f0707fff700 1 --2- 172.21.15.110:0/3744123566 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f070813a380 0x7f070813a7a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.163+0000 7f0707fff700 1 --2- 172.21.15.110:0/3744123566 >> v2:172.21.15.177:3300/0 conn(0x7f07080ae9e0 0x7f07080aee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.163+0000 7f0707fff700 1 -- 172.21.15.110:0/3744123566 >> 172.21.15.110:0/3744123566 conn(0x7f070806f870 msgr2=0x7f07080b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:43.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.163+0000 7f0707fff700 1 -- 172.21.15.110:0/3744123566 shutdown_connections 2024-09-15T03:36:43.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.163+0000 7f0707fff700 1 -- 172.21.15.110:0/3744123566 wait complete. 2024-09-15T03:36:43.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.163+0000 7f0707fff700 1 Processor -- start 2024-09-15T03:36:43.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.163+0000 7f0707fff700 1 -- start start 2024-09-15T03:36:43.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.163+0000 7f0707fff700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f07080a4d70 0x7f0708150200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:43.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.163+0000 7f0707fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f07080ae9e0 0x7f07080a70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:43.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.163+0000 7f0707fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f070813a380 0x7f07080a7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:43.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.163+0000 7f0707fff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07080a9820 con 0x7f070813a380 2024-09-15T03:36:43.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.163+0000 7f0707fff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07080a7890 con 0x7f07080ae9e0 2024-09-15T03:36:43.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.163+0000 7f0707fff700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f070813c320 con 0x7f07080a4d70 2024-09-15T03:36:43.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.163+0000 7f07077fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f070813a380 0x7f07080a7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:43.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.163+0000 7f0706ffd700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f07080a4d70 0x7f0708150200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:43.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.167+0000 7f07077fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f070813a380 0x7f07080a7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37206/0 (socket says 172.21.15.110:37206) 2024-09-15T03:36:43.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.167+0000 7f0706ffd700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f07080a4d70 0x7f0708150200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3301/0 says I am v2:172.21.15.110:44660/0 (socket says 172.21.15.110:44660) 2024-09-15T03:36:43.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.167+0000 7f07077fe700 1 -- 172.21.15.110:0/1867008404 learned_addr learned my addr 172.21.15.110:0/1867008404 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:36:43.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.167+0000 7f06fe7fc700 1 --2- 172.21.15.110:0/1867008404 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f07080ae9e0 0x7f07080a70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:43.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.167+0000 7f07077fe700 1 -- 172.21.15.110:0/1867008404 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f07080a4d70 msgr2=0x7f0708150200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.167+0000 7f07077fe700 1 --2- 172.21.15.110:0/1867008404 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f07080a4d70 0x7f0708150200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.167+0000 7f07077fe700 1 -- 172.21.15.110:0/1867008404 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f07080ae9e0 msgr2=0x7f07080a70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.167+0000 7f07077fe700 1 --2- 172.21.15.110:0/1867008404 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f07080ae9e0 0x7f07080a70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.167+0000 7f07077fe700 1 -- 172.21.15.110:0/1867008404 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0700009ae0 con 0x7f070813a380 2024-09-15T03:36:43.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.167+0000 7f0706ffd700 1 --2- 172.21.15.110:0/1867008404 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f07080a4d70 0x7f0708150200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T03:36:43.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.167+0000 7f07077fe700 1 --2- 172.21.15.110:0/1867008404 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f070813a380 0x7f07080a7d20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0708067f00 tx=0x7f0700018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:43.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.167+0000 7f06ff7fe700 1 -- 172.21.15.110:0/1867008404 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0700007960 con 0x7f070813a380 2024-09-15T03:36:43.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.167+0000 7f06ff7fe700 1 -- 172.21.15.110:0/1867008404 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0700007ae0 con 0x7f070813a380 2024-09-15T03:36:43.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.167+0000 7f0707fff700 1 -- 172.21.15.110:0/1867008404 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0708150c70 con 0x7f070813a380 2024-09-15T03:36:43.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.167+0000 7f06ff7fe700 1 -- 172.21.15.110:0/1867008404 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0700007ea0 con 0x7f070813a380 2024-09-15T03:36:43.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.167+0000 7f0707fff700 1 -- 172.21.15.110:0/1867008404 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07080a8400 con 0x7f070813a380 2024-09-15T03:36:43.174 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.167+0000 7f0707fff700 1 -- 172.21.15.110:0/1867008404 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f070813c4e0 con 0x7f070813a380 2024-09-15T03:36:43.174 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.167+0000 7f06ff7fe700 1 -- 172.21.15.110:0/1867008404 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) v1 ==== 85847+0+0 (secure 0 0 0) 0x7f070001c410 con 0x7f070813a380 2024-09-15T03:36:43.174 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.167+0000 7f06ff7fe700 1 --2- 172.21.15.110:0/1867008404 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f06e8067a80 0x7f06e8069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:43.174 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.167+0000 7f06ff7fe700 1 -- 172.21.15.110:0/1867008404 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5262+0+0 (secure 0 0 0) 0x7f0700087180 con 0x7f070813a380 2024-09-15T03:36:43.174 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.171+0000 7f0706ffd700 1 --2- 172.21.15.110:0/1867008404 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f06e8067a80 0x7f06e8069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:43.174 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.171+0000 7f0706ffd700 1 --2- 172.21.15.110:0/1867008404 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f06e8067a80 0x7f06e8069f30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f06f8001dc0 tx=0x7f06f8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:43.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.179+0000 7f06ff7fe700 1 -- 172.21.15.110:0/1867008404 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f070813c4e0 con 0x7f070813a380 2024-09-15T03:36:43.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.339+0000 7f0707fff700 1 -- 172.21.15.110:0/1867008404 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f07080a7a40 con 0x7f070813a380 2024-09-15T03:36:43.347 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.343+0000 7f06ff7fe700 1 -- 172.21.15.110:0/1867008404 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+13803 (secure 0 0 0) 0x7f07080a7a40 con 0x7f070813a380 2024-09-15T03:36:43.347 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-15T03:36:43.347 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":19,"fsid":"d7bdbc6e-e9f0-40e5-921a-f7c68fa2d8c1","created":"2024-09-15T03:36:28.375926+0000","modified":"2024-09-15T03:36:42.675107+0000","last_up_change":"2024-09-15T03:36:37.618040+0000","last_in_change":"2024-09-15T03:36:30.385381+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T03:36:36.657583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-15T03:36:39.048914+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"09d93c1d-8ec1-42e8-8b59-5e20958830d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":22352},{"type":"v1","addr":"172.21.15.110:6801","nonce":22352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":22352},{"type":"v1","addr":"172.21.15.110:6803","nonce":22352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":22352},{"type":"v1","addr":"172.21.15.110:6807","nonce":22352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":22352},{"type":"v1","addr":"172.21.15.110:6805","nonce":22352}]},"public_addr":"172.21.15.110:6801/22352","cluster_addr":"172.21.15.110:6803/22352","heartbeat_back_addr":"172.21.15.110:6807/22352","heartbeat_front_addr":"172.21.15.110:6805/22352","state":["exists","up"]},{"osd":1,"uuid":"378acca0-6784-42c1-a6f8-0ba39a287db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":22354},{"type":"v1","addr":"172.21.15.110:6809","nonce":22354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":22354},{"type":"v1","addr":"172.21.15.110:6811","nonce":22354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":22354},{"type":"v1","addr":"172.21.15.110:6815","nonce":22354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":22354},{"type":"v1","addr":"172.21.15.110:6813","nonce":22354}]},"public_addr":"172.21.15.110:6809/22354","cluster_addr":"172.21.15.110:6811/22354","heartbeat_back_addr":"172.21.15.110:6815/22354","heartbeat_front_addr":"172.21.15.110:6813/22354","state":["exists","up"]},{"osd":2,"uuid":"b1c2c060-985a-429c-bf05-ccfd5fdf3d8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":22356},{"type":"v1","addr":"172.21.15.110:6824","nonce":22356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":22356},{"type":"v1","addr":"172.21.15.110:6827","nonce":22356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":22356},{"type":"v1","addr":"172.21.15.110:6831","nonce":22356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":22356},{"type":"v1","addr":"172.21.15.110:6829","nonce":22356}]},"public_addr":"172.21.15.110:6824/22356","cluster_addr":"172.21.15.110:6827/22356","heartbeat_back_addr":"172.21.15.110:6831/22356","heartbeat_front_addr":"172.21.15.110:6829/22356","state":["exists","up"]},{"osd":3,"uuid":"aca55104-d76b-4719-8837-252c2e55d0cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":22355},{"type":"v1","addr":"172.21.15.110:6817","nonce":22355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":22355},{"type":"v1","addr":"172.21.15.110:6819","nonce":22355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6823","nonce":22355},{"type":"v1","addr":"172.21.15.110:6825","nonce":22355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":22355},{"type":"v1","addr":"172.21.15.110:6821","nonce":22355}]},"public_addr":"172.21.15.110:6817/22355","cluster_addr":"172.21.15.110:6819/22355","heartbeat_back_addr":"172.21.15.110:6825/22355","heartbeat_front_addr":"172.21.15.110:6821/22355","state":["exists","up"]},{"osd":4,"uuid":"7a87be3c-b6ff-43a4-8672-39d3006a0e2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":22425},{"type":"v1","addr":"172.21.15.177:6824","nonce":22425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":22425},{"type":"v1","addr":"172.21.15.177:6827","nonce":22425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":22425},{"type":"v1","addr":"172.21.15.177:6831","nonce":22425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":22425},{"type":"v1","addr":"172.21.15.177:6829","nonce":22425}]},"public_addr":"172.21.15.177:6824/22425","cluster_addr":"172.21.15.177:6827/22425","heartbeat_back_addr":"172.21.15.177:6831/22425","heartbeat_front_addr":"172.21.15.177:6829/22425","state":["exists","up"]},{"osd":5,"uuid":"a8f3a63d-45af-41fe-be33-2339d7b9ed96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":22424},{"type":"v1","addr":"172.21.15.177:6804","nonce":22424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":22424},{"type":"v1","addr":"172.21.15.177:6808","nonce":22424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":22424},{"type":"v1","addr":"172.21.15.177:6815","nonce":22424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":22424},{"type":"v1","addr":"172.21.15.177:6812","nonce":22424}]},"public_addr":"172.21.15.177:6804/22424","cluster_addr":"172.21.15.177:6808/22424","heartbeat_back_addr":"172.21.15.177:6815/22424","heartbeat_front_addr":"172.21.15.177:6812/22424","state":["exists","up"]},{"osd":6,"uuid":"a61b52dd-273d-47b4-bf92-0610cce86e1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":22428},{"type":"v1","addr":"172.21.15.177:6817","nonce":22428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":22428},{"type":"v1","addr":"172.21.15.177:6819","nonce":22428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":22428},{"type":"v1","addr":"172.21.15.177:6825","nonce":22428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":22428},{"type":"v1","addr":"172.21.15.177:6821","nonce":22428}]},"public_addr":"172.21.15.177:6817/22428","cluster_addr":"172.21.15.177:6819/22428","heartbeat_back_addr":"172.21.15.177:6825/22428","heartbeat_front_addr":"172.21.15.177:6821/22428","state":["exists","up"]},{"osd":7,"uuid":"1a7d5a9f-6b41-47e1-b0ef-45f4d638fd65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":22423},{"type":"v1","addr":"172.21.15.177:6801","nonce":22423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6803","nonce":22423},{"type":"v1","addr":"172.21.15.177:6805","nonce":22423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6811","nonce":22423},{"type":"v1","addr":"172.21.15.177:6813","nonce":22423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6807","nonce":22423},{"type":"v1","addr":"172.21.15.177:6809","nonce":22423}]},"public_addr":"172.21.15.177:6801/22423","cluster_addr":"172.21.15.177:6805/22423","heartbeat_back_addr":"172.21.15.177:6813/22423","heartbeat_front_addr":"172.21.15.177:6809/22423","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:36:35.106295+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:36:35.495899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:36:35.902086+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:36:35.652162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:36:35.645490+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:36:35.658420+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:36:35.622386+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:36:35.604615+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T03:36:43.349 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.343+0000 7f0707fff700 1 -- 172.21.15.110:0/1867008404 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f06e8067a80 msgr2=0x7f06e8069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.349 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.343+0000 7f0707fff700 1 --2- 172.21.15.110:0/1867008404 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f06e8067a80 0x7f06e8069f30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f06f8001dc0 tx=0x7f06f8009040 comp rx=0 tx=0).stop 2024-09-15T03:36:43.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.343+0000 7f0707fff700 1 -- 172.21.15.110:0/1867008404 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f070813a380 msgr2=0x7f07080a7d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.343+0000 7f0707fff700 1 --2- 172.21.15.110:0/1867008404 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f070813a380 0x7f07080a7d20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0708067f00 tx=0x7f0700018000 comp rx=0 tx=0).stop 2024-09-15T03:36:43.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.343+0000 7f0707fff700 1 -- 172.21.15.110:0/1867008404 shutdown_connections 2024-09-15T03:36:43.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.343+0000 7f0707fff700 1 --2- 172.21.15.110:0/1867008404 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f06e8067a80 0x7f06e8069f30 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.343+0000 7f0707fff700 1 --2- 172.21.15.110:0/1867008404 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f070813a380 0x7f07080a7d20 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.343+0000 7f0707fff700 1 --2- 172.21.15.110:0/1867008404 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f07080ae9e0 0x7f07080a70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.343+0000 7f0707fff700 1 --2- 172.21.15.110:0/1867008404 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f07080a4d70 0x7f0708150200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.343+0000 7f0707fff700 1 -- 172.21.15.110:0/1867008404 >> 172.21.15.110:0/1867008404 conn(0x7f070806f870 msgr2=0x7f07080b1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:43.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.343+0000 7f0707fff700 1 -- 172.21.15.110:0/1867008404 shutdown_connections 2024-09-15T03:36:43.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.343+0000 7f0707fff700 1 -- 172.21.15.110:0/1867008404 wait complete. 2024-09-15T03:36:43.366 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-15T03:36:43.367 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T03:36:43.473 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.467+0000 7fe701174700 1 Processor -- start 2024-09-15T03:36:43.473 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.467+0000 7fe701174700 1 -- start start 2024-09-15T03:36:43.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.471+0000 7fe701174700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fe6fc0aeae0 0x7fe6fc0aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:43.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.471+0000 7fe701174700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fe6fc13a620 0x7fe6fc13aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:43.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.471+0000 7fe701174700 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6fc0a6850 con 0x7fe6fc0a4d70 2024-09-15T03:36:43.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.471+0000 7fe701174700 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6fc0af430 con 0x7fe6fc13a620 2024-09-15T03:36:43.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.471+0000 7fe701174700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6fc0a5140 con 0x7fe6fc0aeae0 2024-09-15T03:36:43.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.471+0000 7fe6fa59c700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fe6fc0aeae0 0x7fe6fc0aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:43.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.471+0000 7fe6fa59c700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fe6fc0aeae0 0x7fe6fc0aeef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3301/0 says I am v2:172.21.15.110:44668/0 (socket says 172.21.15.110:44668) 2024-09-15T03:36:43.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.471+0000 7fe6fa59c700 1 -- 172.21.15.110:0/1967316046 learned_addr learned my addr 172.21.15.110:0/1967316046 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:36:43.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.471+0000 7fe6fb59e700 1 --2- 172.21.15.110:0/1967316046 >> v2:172.21.15.110:3300/0 conn(0x7fe6fc13a620 0x7fe6fc13aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:43.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.471+0000 7fe6fa59c700 1 -- 172.21.15.110:0/1967316046 >> v2:172.21.15.110:3300/0 conn(0x7fe6fc13a620 msgr2=0x7fe6fc13aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:36:43.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.471+0000 7fe6fa59c700 1 --2- 172.21.15.110:0/1967316046 >> v2:172.21.15.110:3300/0 conn(0x7fe6fc13a620 0x7fe6fc13aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.471+0000 7fe6fa59c700 1 -- 172.21.15.110:0/1967316046 >> v1:172.21.15.110:6789/0 conn(0x7fe6fc0a4d70 legacy=0x7fe6fc140d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.471+0000 7fe6fa59c700 1 -- 172.21.15.110:0/1967316046 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6fc1415c0 con 0x7fe6fc0aeae0 2024-09-15T03:36:43.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.471+0000 7fe6fa59c700 1 --2- 172.21.15.110:0/1967316046 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fe6fc0aeae0 0x7fe6fc0aeef0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4009f20 tx=0x7fe6f4009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43d71162aa11ba28 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:43.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.471+0000 7fe6f9d9b700 1 -- 172.21.15.110:0/1967316046 <== mon.2 v2:172.21.15.110:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f400e070 con 0x7fe6fc0aeae0 2024-09-15T03:36:43.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.471+0000 7fe6f9d9b700 1 -- 172.21.15.110:0/1967316046 <== mon.2 v2:172.21.15.110:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6f400ba80 con 0x7fe6fc0aeae0 2024-09-15T03:36:43.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.471+0000 7fe6f9d9b700 1 -- 172.21.15.110:0/1967316046 <== mon.2 v2:172.21.15.110:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f40092e0 con 0x7fe6fc0aeae0 2024-09-15T03:36:43.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.471+0000 7fe701174700 1 -- 172.21.15.110:0/1967316046 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fe6fc0aeae0 msgr2=0x7fe6fc0aeef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.471+0000 7fe701174700 1 --2- 172.21.15.110:0/1967316046 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fe6fc0aeae0 0x7fe6fc0aeef0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4009f20 tx=0x7fe6f4009840 comp rx=0 tx=0).stop 2024-09-15T03:36:43.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.471+0000 7fe701174700 1 -- 172.21.15.110:0/1967316046 shutdown_connections 2024-09-15T03:36:43.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.471+0000 7fe701174700 1 --2- 172.21.15.110:0/1967316046 >> v2:172.21.15.110:3300/0 conn(0x7fe6fc13a620 0x7fe6fc13aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.471+0000 7fe701174700 1 --2- 172.21.15.110:0/1967316046 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fe6fc0aeae0 0x7fe6fc0aeef0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.471+0000 7fe701174700 1 -- 172.21.15.110:0/1967316046 >> 172.21.15.110:0/1967316046 conn(0x7fe6fc06f870 msgr2=0x7fe6fc0b4170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:43.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.471+0000 7fe701174700 1 -- 172.21.15.110:0/1967316046 shutdown_connections 2024-09-15T03:36:43.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.471+0000 7fe701174700 1 -- 172.21.15.110:0/1967316046 wait complete. 2024-09-15T03:36:43.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.475+0000 7fe701174700 1 Processor -- start 2024-09-15T03:36:43.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.475+0000 7fe701174700 1 -- start start 2024-09-15T03:36:43.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.475+0000 7fe701174700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6fc0a4d70 0x7fe6fc144630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:43.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.475+0000 7fe701174700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fe6fc0aeae0 0x7fe6fc13cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:43.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.475+0000 7fe701174700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6fc13a620 0x7fe6fc13dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:43.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.475+0000 7fe701174700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6fc13d870 con 0x7fe6fc0a4d70 2024-09-15T03:36:43.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.475+0000 7fe701174700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6fc0a7e30 con 0x7fe6fc13a620 2024-09-15T03:36:43.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.475+0000 7fe701174700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6fc1450f0 con 0x7fe6fc0aeae0 2024-09-15T03:36:43.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.475+0000 7fe6fa59c700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fe6fc0aeae0 0x7fe6fc13cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:43.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.475+0000 7fe6fad9d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6fc0a4d70 0x7fe6fc144630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:43.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.475+0000 7fe6fa59c700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fe6fc0aeae0 0x7fe6fc13cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3301/0 says I am v2:172.21.15.110:44680/0 (socket says 172.21.15.110:44680) 2024-09-15T03:36:43.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.475+0000 7fe6fa59c700 1 -- 172.21.15.110:0/994733711 learned_addr learned my addr 172.21.15.110:0/994733711 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:36:43.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.475+0000 7fe6fb59e700 1 --2- 172.21.15.110:0/994733711 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6fc13a620 0x7fe6fc13dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:43.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.475+0000 7fe6fa59c700 1 -- 172.21.15.110:0/994733711 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6fc13a620 msgr2=0x7fe6fc13dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.475+0000 7fe6fa59c700 1 --2- 172.21.15.110:0/994733711 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6fc13a620 0x7fe6fc13dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.475+0000 7fe6fa59c700 1 -- 172.21.15.110:0/994733711 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6fc0a4d70 msgr2=0x7fe6fc144630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.475+0000 7fe6fa59c700 1 --2- 172.21.15.110:0/994733711 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6fc0a4d70 0x7fe6fc144630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.475+0000 7fe6fa59c700 1 -- 172.21.15.110:0/994733711 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6f4009ae0 con 0x7fe6fc0aeae0 2024-09-15T03:36:43.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.475+0000 7fe6fa59c700 1 --2- 172.21.15.110:0/994733711 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fe6fc0aeae0 0x7fe6fc13cf20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4009430 tx=0x7fe6f4018000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:43.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.475+0000 7fe6e2ffd700 1 -- 172.21.15.110:0/994733711 <== mon.2 v2:172.21.15.110:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f40094e0 con 0x7fe6fc0aeae0 2024-09-15T03:36:43.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.475+0000 7fe6e2ffd700 1 -- 172.21.15.110:0/994733711 <== mon.2 v2:172.21.15.110:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6f40088b0 con 0x7fe6fc0aeae0 2024-09-15T03:36:43.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.475+0000 7fe701174700 1 -- 172.21.15.110:0/994733711 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6fc13d460 con 0x7fe6fc0aeae0 2024-09-15T03:36:43.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.475+0000 7fe6e2ffd700 1 -- 172.21.15.110:0/994733711 <== mon.2 v2:172.21.15.110:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f4008c70 con 0x7fe6fc0aeae0 2024-09-15T03:36:43.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.475+0000 7fe701174700 1 -- 172.21.15.110:0/994733711 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6fc13e470 con 0x7fe6fc0aeae0 2024-09-15T03:36:43.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.479+0000 7fe701174700 1 -- 172.21.15.110:0/994733711 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6fc13d9e0 con 0x7fe6fc0aeae0 2024-09-15T03:36:43.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.479+0000 7fe6e2ffd700 1 -- 172.21.15.110:0/994733711 <== mon.2 v2:172.21.15.110:3301/0 4 ==== mgrmap(e 5) v1 ==== 85847+0+0 (secure 0 0 0) 0x7fe6f4003040 con 0x7fe6fc0aeae0 2024-09-15T03:36:43.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.479+0000 7fe6e2ffd700 1 --2- 172.21.15.110:0/994733711 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fe6e8067a80 0x7fe6e8069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:43.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.479+0000 7fe6e2ffd700 1 -- 172.21.15.110:0/994733711 <== mon.2 v2:172.21.15.110:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5262+0+0 (secure 0 0 0) 0x7fe6f4087300 con 0x7fe6fc0aeae0 2024-09-15T03:36:43.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.479+0000 7fe6fad9d700 1 --2- 172.21.15.110:0/994733711 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fe6e8067a80 0x7fe6e8069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:43.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.479+0000 7fe6fad9d700 1 --2- 172.21.15.110:0/994733711 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fe6e8067a80 0x7fe6e8069f30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe6f0003aa0 tx=0x7fe6f0000bc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:43.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.487+0000 7fe6e2ffd700 1 -- 172.21.15.110:0/994733711 <== mon.2 v2:172.21.15.110:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fe6fc13d9e0 con 0x7fe6fc0aeae0 2024-09-15T03:36:43.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.651+0000 7fe701174700 1 -- 172.21.15.110:0/994733711 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe6fc13d9e0 con 0x7fe6fc0aeae0 2024-09-15T03:36:43.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.655+0000 7fe6e2ffd700 1 -- 172.21.15.110:0/994733711 <== mon.2 v2:172.21.15.110:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+13803 (secure 0 0 0) 0x7fe6fc13d9e0 con 0x7fe6fc0aeae0 2024-09-15T03:36:43.659 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-15T03:36:43.659 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":19,"fsid":"d7bdbc6e-e9f0-40e5-921a-f7c68fa2d8c1","created":"2024-09-15T03:36:28.375926+0000","modified":"2024-09-15T03:36:42.675107+0000","last_up_change":"2024-09-15T03:36:37.618040+0000","last_in_change":"2024-09-15T03:36:30.385381+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T03:36:36.657583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-15T03:36:39.048914+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"09d93c1d-8ec1-42e8-8b59-5e20958830d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":22352},{"type":"v1","addr":"172.21.15.110:6801","nonce":22352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":22352},{"type":"v1","addr":"172.21.15.110:6803","nonce":22352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":22352},{"type":"v1","addr":"172.21.15.110:6807","nonce":22352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":22352},{"type":"v1","addr":"172.21.15.110:6805","nonce":22352}]},"public_addr":"172.21.15.110:6801/22352","cluster_addr":"172.21.15.110:6803/22352","heartbeat_back_addr":"172.21.15.110:6807/22352","heartbeat_front_addr":"172.21.15.110:6805/22352","state":["exists","up"]},{"osd":1,"uuid":"378acca0-6784-42c1-a6f8-0ba39a287db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":22354},{"type":"v1","addr":"172.21.15.110:6809","nonce":22354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":22354},{"type":"v1","addr":"172.21.15.110:6811","nonce":22354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":22354},{"type":"v1","addr":"172.21.15.110:6815","nonce":22354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":22354},{"type":"v1","addr":"172.21.15.110:6813","nonce":22354}]},"public_addr":"172.21.15.110:6809/22354","cluster_addr":"172.21.15.110:6811/22354","heartbeat_back_addr":"172.21.15.110:6815/22354","heartbeat_front_addr":"172.21.15.110:6813/22354","state":["exists","up"]},{"osd":2,"uuid":"b1c2c060-985a-429c-bf05-ccfd5fdf3d8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":22356},{"type":"v1","addr":"172.21.15.110:6824","nonce":22356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":22356},{"type":"v1","addr":"172.21.15.110:6827","nonce":22356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":22356},{"type":"v1","addr":"172.21.15.110:6831","nonce":22356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":22356},{"type":"v1","addr":"172.21.15.110:6829","nonce":22356}]},"public_addr":"172.21.15.110:6824/22356","cluster_addr":"172.21.15.110:6827/22356","heartbeat_back_addr":"172.21.15.110:6831/22356","heartbeat_front_addr":"172.21.15.110:6829/22356","state":["exists","up"]},{"osd":3,"uuid":"aca55104-d76b-4719-8837-252c2e55d0cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":22355},{"type":"v1","addr":"172.21.15.110:6817","nonce":22355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":22355},{"type":"v1","addr":"172.21.15.110:6819","nonce":22355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6823","nonce":22355},{"type":"v1","addr":"172.21.15.110:6825","nonce":22355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":22355},{"type":"v1","addr":"172.21.15.110:6821","nonce":22355}]},"public_addr":"172.21.15.110:6817/22355","cluster_addr":"172.21.15.110:6819/22355","heartbeat_back_addr":"172.21.15.110:6825/22355","heartbeat_front_addr":"172.21.15.110:6821/22355","state":["exists","up"]},{"osd":4,"uuid":"7a87be3c-b6ff-43a4-8672-39d3006a0e2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":22425},{"type":"v1","addr":"172.21.15.177:6824","nonce":22425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":22425},{"type":"v1","addr":"172.21.15.177:6827","nonce":22425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":22425},{"type":"v1","addr":"172.21.15.177:6831","nonce":22425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":22425},{"type":"v1","addr":"172.21.15.177:6829","nonce":22425}]},"public_addr":"172.21.15.177:6824/22425","cluster_addr":"172.21.15.177:6827/22425","heartbeat_back_addr":"172.21.15.177:6831/22425","heartbeat_front_addr":"172.21.15.177:6829/22425","state":["exists","up"]},{"osd":5,"uuid":"a8f3a63d-45af-41fe-be33-2339d7b9ed96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":22424},{"type":"v1","addr":"172.21.15.177:6804","nonce":22424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":22424},{"type":"v1","addr":"172.21.15.177:6808","nonce":22424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":22424},{"type":"v1","addr":"172.21.15.177:6815","nonce":22424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":22424},{"type":"v1","addr":"172.21.15.177:6812","nonce":22424}]},"public_addr":"172.21.15.177:6804/22424","cluster_addr":"172.21.15.177:6808/22424","heartbeat_back_addr":"172.21.15.177:6815/22424","heartbeat_front_addr":"172.21.15.177:6812/22424","state":["exists","up"]},{"osd":6,"uuid":"a61b52dd-273d-47b4-bf92-0610cce86e1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":22428},{"type":"v1","addr":"172.21.15.177:6817","nonce":22428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":22428},{"type":"v1","addr":"172.21.15.177:6819","nonce":22428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":22428},{"type":"v1","addr":"172.21.15.177:6825","nonce":22428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":22428},{"type":"v1","addr":"172.21.15.177:6821","nonce":22428}]},"public_addr":"172.21.15.177:6817/22428","cluster_addr":"172.21.15.177:6819/22428","heartbeat_back_addr":"172.21.15.177:6825/22428","heartbeat_front_addr":"172.21.15.177:6821/22428","state":["exists","up"]},{"osd":7,"uuid":"1a7d5a9f-6b41-47e1-b0ef-45f4d638fd65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":22423},{"type":"v1","addr":"172.21.15.177:6801","nonce":22423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6803","nonce":22423},{"type":"v1","addr":"172.21.15.177:6805","nonce":22423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6811","nonce":22423},{"type":"v1","addr":"172.21.15.177:6813","nonce":22423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6807","nonce":22423},{"type":"v1","addr":"172.21.15.177:6809","nonce":22423}]},"public_addr":"172.21.15.177:6801/22423","cluster_addr":"172.21.15.177:6805/22423","heartbeat_back_addr":"172.21.15.177:6813/22423","heartbeat_front_addr":"172.21.15.177:6809/22423","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:36:35.106295+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:36:35.495899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:36:35.902086+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:36:35.652162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:36:35.645490+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:36:35.658420+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:36:35.622386+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:36:35.604615+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T03:36:43.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.655+0000 7fe701174700 1 -- 172.21.15.110:0/994733711 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fe6e8067a80 msgr2=0x7fe6e8069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.655+0000 7fe701174700 1 --2- 172.21.15.110:0/994733711 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fe6e8067a80 0x7fe6e8069f30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe6f0003aa0 tx=0x7fe6f0000bc0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.655+0000 7fe701174700 1 -- 172.21.15.110:0/994733711 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fe6fc0aeae0 msgr2=0x7fe6fc13cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.655+0000 7fe701174700 1 --2- 172.21.15.110:0/994733711 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fe6fc0aeae0 0x7fe6fc13cf20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4009430 tx=0x7fe6f4018000 comp rx=0 tx=0).stop 2024-09-15T03:36:43.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.655+0000 7fe701174700 1 -- 172.21.15.110:0/994733711 shutdown_connections 2024-09-15T03:36:43.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.655+0000 7fe701174700 1 --2- 172.21.15.110:0/994733711 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fe6e8067a80 0x7fe6e8069f30 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.655+0000 7fe701174700 1 --2- 172.21.15.110:0/994733711 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6fc13a620 0x7fe6fc13dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.655+0000 7fe701174700 1 --2- 172.21.15.110:0/994733711 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fe6fc0aeae0 0x7fe6fc13cf20 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.655+0000 7fe701174700 1 --2- 172.21.15.110:0/994733711 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6fc0a4d70 0x7fe6fc144630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.655+0000 7fe701174700 1 -- 172.21.15.110:0/994733711 >> 172.21.15.110:0/994733711 conn(0x7fe6fc06f870 msgr2=0x7fe6fc139a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:43.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.655+0000 7fe701174700 1 -- 172.21.15.110:0/994733711 shutdown_connections 2024-09-15T03:36:43.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.655+0000 7fe701174700 1 -- 172.21.15.110:0/994733711 wait complete. 2024-09-15T03:36:43.673 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-15T03:36:43.674 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-15T03:36:43.674 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-15T03:36:43.674 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-15T03:36:43.675 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-15T03:36:43.675 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-15T03:36:43.675 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-15T03:36:43.676 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-15T03:36:43.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.811+0000 7f94fceea700 1 Processor -- start 2024-09-15T03:36:43.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.811+0000 7f94fceea700 1 -- start start 2024-09-15T03:36:43.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.811+0000 7f94fceea700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f94f813d120 0x7f94f813d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:43.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.811+0000 7f94fceea700 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94f806cb00 con 0x7f94f8139bc0 2024-09-15T03:36:43.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.811+0000 7f94fceea700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94f806d860 con 0x7f94f80aced0 2024-09-15T03:36:43.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.811+0000 7f94fceea700 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94f80ac6e0 con 0x7f94f813d120 2024-09-15T03:36:43.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.811+0000 7f94f5d9b700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f94f813d120 0x7f94f813d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:43.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.811+0000 7f94f5d9b700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f94f813d120 0x7f94f813d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37224/0 (socket says 172.21.15.110:37224) 2024-09-15T03:36:43.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.811+0000 7f94f5d9b700 1 -- 172.21.15.110:0/2837989937 learned_addr learned my addr 172.21.15.110:0/2837989937 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:36:43.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.811+0000 7f94f5d9b700 1 -- 172.21.15.110:0/2837989937 >> v1:172.21.15.177:6789/0 conn(0x7f94f80aced0 legacy=0x7f94f806c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.811+0000 7f94f5d9b700 1 -- 172.21.15.110:0/2837989937 >> v1:172.21.15.110:6789/0 conn(0x7f94f8139bc0 legacy=0x7f94f813a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.811+0000 7f94f5d9b700 1 -- 172.21.15.110:0/2837989937 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94f80ad2a0 con 0x7f94f813d120 2024-09-15T03:36:43.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.811+0000 7f94f5d9b700 1 --2- 172.21.15.110:0/2837989937 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f94f813d120 0x7f94f813d550 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f94f000d1a0 tx=0x7f94f0007690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a58a43329a5cfb17 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:43.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.811+0000 7f0356d72700 1 Processor -- start 2024-09-15T03:36:43.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.811+0000 7f0356d72700 1 -- start start 2024-09-15T03:36:43.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.811+0000 7f94f559a700 1 -- 172.21.15.110:0/2837989937 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f0010070 con 0x7f94f813d120 2024-09-15T03:36:43.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.811+0000 7f94f559a700 1 -- 172.21.15.110:0/2837989937 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94f000dcb0 con 0x7f94f813d120 2024-09-15T03:36:43.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.811+0000 7f0356d72700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f03500aced0 0x7f035006c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:43.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.811+0000 7f0356d72700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f035013d120 0x7f035013d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:43.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.811+0000 7f0356d72700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f0350068180 0x7f03500ab030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:43.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.811+0000 7f0356d72700 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f035006cb00 con 0x7f03500aced0 2024-09-15T03:36:43.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.811+0000 7f0356d72700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f035006c7c0 con 0x7f035013d120 2024-09-15T03:36:43.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.811+0000 7f0356d72700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f035006d860 con 0x7f0350068180 2024-09-15T03:36:43.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.811+0000 7f0354b0e700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f03500aced0 0x7f035006c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:43.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.811+0000 7f0354b0e700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f03500aced0 0x7f035006c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37226/0 (socket says 172.21.15.110:37226) 2024-09-15T03:36:43.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.815+0000 7f0354b0e700 1 -- 172.21.15.110:0/2606302270 learned_addr learned my addr 172.21.15.110:0/2606302270 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:36:43.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.815+0000 7f034ffff700 1 --2- 172.21.15.110:0/2606302270 >> v2:172.21.15.177:3300/0 conn(0x7f035013d120 0x7f035013d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:43.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.815+0000 7f94fceea700 1 -- 172.21.15.110:0/2837989937 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f94f813d120 msgr2=0x7f94f813d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.815+0000 7f94fceea700 1 --2- 172.21.15.110:0/2837989937 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f94f813d120 0x7f94f813d550 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f94f000d1a0 tx=0x7f94f0007690 comp rx=0 tx=0).stop 2024-09-15T03:36:43.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.815+0000 7f0354b0e700 1 -- 172.21.15.110:0/2606302270 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f0350068180 msgr2=0x7f03500ab030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:36:43.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.815+0000 7f035530f700 1 --2- 172.21.15.110:0/2606302270 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f0350068180 0x7f03500ab030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:43.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.815+0000 7f94fceea700 1 -- 172.21.15.110:0/2837989937 shutdown_connections 2024-09-15T03:36:43.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.815+0000 7f94fceea700 1 --2- 172.21.15.110:0/2837989937 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f94f813d120 0x7f94f813d550 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.815+0000 7f94fceea700 1 -- 172.21.15.110:0/2837989937 >> 172.21.15.110:0/2837989937 conn(0x7f94f8067670 msgr2=0x7f94f8067a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:43.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.815+0000 7f0354b0e700 1 --2- 172.21.15.110:0/2606302270 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f0350068180 0x7f03500ab030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.815+0000 7f0354b0e700 1 -- 172.21.15.110:0/2606302270 >> v2:172.21.15.177:3300/0 conn(0x7f035013d120 msgr2=0x7f035013d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:36:43.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.815+0000 7f0354b0e700 1 --2- 172.21.15.110:0/2606302270 >> v2:172.21.15.177:3300/0 conn(0x7f035013d120 0x7f035013d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.815+0000 7f0354b0e700 1 -- 172.21.15.110:0/2606302270 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03500ab570 con 0x7f03500aced0 2024-09-15T03:36:43.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.815+0000 7f94fceea700 1 -- 172.21.15.110:0/2837989937 shutdown_connections 2024-09-15T03:36:43.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.815+0000 7f94fceea700 1 -- 172.21.15.110:0/2837989937 wait complete. 2024-09-15T03:36:43.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.815+0000 7f035530f700 1 --2- 172.21.15.110:0/2606302270 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f0350068180 0x7f03500ab030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T03:36:43.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.815+0000 7f0354b0e700 1 --2- 172.21.15.110:0/2606302270 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03500aced0 0x7f035006c280 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f03500abf50 tx=0x7f0340009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=add9c35cafed8dc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:43.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.815+0000 7f034ffff700 1 --2- 172.21.15.110:0/2606302270 >> v2:172.21.15.177:3300/0 conn(0x7f035013d120 0x7f035013d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T03:36:43.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.815+0000 7f034f7fe700 1 -- 172.21.15.110:0/2606302270 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0340009e70 con 0x7f03500aced0 2024-09-15T03:36:43.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.815+0000 7f034f7fe700 1 -- 172.21.15.110:0/2606302270 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f034000b860 con 0x7f03500aced0 2024-09-15T03:36:43.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.815+0000 7f94fceea700 1 Processor -- start 2024-09-15T03:36:43.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.815+0000 7f94fceea700 1 -- start start 2024-09-15T03:36:43.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.815+0000 7f0356d72700 1 -- 172.21.15.110:0/2606302270 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03500aced0 msgr2=0x7f035006c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.815+0000 7f0356d72700 1 --2- 172.21.15.110:0/2606302270 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03500aced0 0x7f035006c280 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f03500abf50 tx=0x7f0340009250 comp rx=0 tx=0).stop 2024-09-15T03:36:43.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.815+0000 7f0356d72700 1 -- 172.21.15.110:0/2606302270 shutdown_connections 2024-09-15T03:36:43.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.815+0000 7f0356d72700 1 --2- 172.21.15.110:0/2606302270 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f0350068180 0x7f03500ab030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.815+0000 7f0356d72700 1 --2- 172.21.15.110:0/2606302270 >> v2:172.21.15.177:3300/0 conn(0x7f035013d120 0x7f035013d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.815+0000 7f0356d72700 1 --2- 172.21.15.110:0/2606302270 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03500aced0 0x7f035006c280 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.815+0000 7f0356d72700 1 -- 172.21.15.110:0/2606302270 >> 172.21.15.110:0/2606302270 conn(0x7f0350067670 msgr2=0x7f0350067a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:43.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.815+0000 7f0356d72700 1 -- 172.21.15.110:0/2606302270 shutdown_connections 2024-09-15T03:36:43.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.815+0000 7f0356d72700 1 -- 172.21.15.110:0/2606302270 wait complete. 2024-09-15T03:36:43.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.815+0000 7f94fceea700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f94f80aced0 0x7f94f80b0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:43.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.815+0000 7f94fceea700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f94f8139bc0 0x7f94f814c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:43.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.815+0000 7f94fceea700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f94f813d120 0x7f94f813abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:43.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.815+0000 7f94fceea700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f80afa20 con 0x7f94f813d120 2024-09-15T03:36:43.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.815+0000 7f94fceea700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f80aeb70 con 0x7f94f8139bc0 2024-09-15T03:36:43.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.815+0000 7f94fceea700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94f813bf70 con 0x7f94f80aced0 2024-09-15T03:36:43.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.815+0000 7fee9974a700 1 Processor -- start 2024-09-15T03:36:43.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.815+0000 7f94f6d9d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f94f813d120 0x7f94f813abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:43.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.815+0000 7f94f6d9d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f94f813d120 0x7f94f813abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37234/0 (socket says 172.21.15.110:37234) 2024-09-15T03:36:43.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.815+0000 7f94f6d9d700 1 -- 172.21.15.110:0/453878828 learned_addr learned my addr 172.21.15.110:0/453878828 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:36:43.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.815+0000 7f0356d72700 1 Processor -- start 2024-09-15T03:36:43.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7f0356d72700 1 -- start start 2024-09-15T03:36:43.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.815+0000 7f94f659c700 1 --2- 172.21.15.110:0/453878828 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f94f80aced0 0x7f94f80b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:43.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.815+0000 7fee9974a700 1 -- start start 2024-09-15T03:36:43.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7fee9974a700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fee940678a0 0x7fee94139850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:43.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7fee9974a700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fee94139d90 0x7fee9413cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:43.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7fee9974a700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee9406f220 con 0x7fee9413a1f0 2024-09-15T03:36:43.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7fee9974a700 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee9406dbc0 con 0x7fee940678a0 2024-09-15T03:36:43.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7f94f6d9d700 1 -- 172.21.15.110:0/453878828 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f94f80aced0 msgr2=0x7f94f80b0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7f94f6d9d700 1 --2- 172.21.15.110:0/453878828 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f94f80aced0 0x7f94f80b0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7f94f6d9d700 1 -- 172.21.15.110:0/453878828 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f94f8139bc0 msgr2=0x7f94f814c020 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-15T03:36:43.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7f94f6d9d700 1 --2- 172.21.15.110:0/453878828 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f94f8139bc0 0x7f94f814c020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7f94f6d9d700 1 -- 172.21.15.110:0/453878828 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94f000be70 con 0x7f94f813d120 2024-09-15T03:36:43.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7fee9974a700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee94067c70 con 0x7fee94139d90 2024-09-15T03:36:43.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7fee92ffd700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fee940678a0 0x7fee94139850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:43.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7fee92ffd700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fee940678a0 0x7fee94139850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37256/0 (socket says 172.21.15.110:37256) 2024-09-15T03:36:43.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7fee92ffd700 1 -- 172.21.15.110:0/3041231440 learned_addr learned my addr 172.21.15.110:0/3041231440 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:36:43.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7fee6627e700 1 Processor -- start 2024-09-15T03:36:43.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7f0356d72700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f0350068180 0x7f0350141a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:43.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7fee6627e700 1 -- start start 2024-09-15T03:36:43.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7f0356d72700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03500aced0 0x7f035014d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:43.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7f0356d72700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f035013d120 0x7f0350150140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:43.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7f0356d72700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f035014cce0 con 0x7f03500aced0 2024-09-15T03:36:43.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7f0356d72700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03501405f0 con 0x7f035013d120 2024-09-15T03:36:43.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7f0356d72700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0350141fc0 con 0x7f0350068180 2024-09-15T03:36:43.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7f94f6d9d700 1 --2- 172.21.15.110:0/453878828 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f94f813d120 0x7f94f813abd0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f94e800c560 tx=0x7f94e8007eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:43.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7f0354b0e700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f0350068180 0x7f0350141a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:43.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7f0354b0e700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f0350068180 0x7f0350141a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3301/0 says I am v2:172.21.15.110:44694/0 (socket says 172.21.15.110:44694) 2024-09-15T03:36:43.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7f0354b0e700 1 -- 172.21.15.110:0/4138983868 learned_addr learned my addr 172.21.15.110:0/4138983868 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:36:43.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7f035530f700 1 --2- 172.21.15.110:0/4138983868 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f035013d120 0x7f0350150140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:43.834 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7f034ffff700 1 --2- 172.21.15.110:0/4138983868 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03500aced0 0x7f035014d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:43.834 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7fee92ffd700 1 -- 172.21.15.110:0/3041231440 >> v2:172.21.15.177:3300/0 conn(0x7fee94139d90 msgr2=0x7fee9413cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:36:43.834 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7fee91ffb700 1 -- 172.21.15.110:0/3041231440 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 328812191 0 0) 0x7fee9406f220 con 0x7fee9413a1f0 2024-09-15T03:36:43.834 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7f94e67fc700 1 -- 172.21.15.110:0/453878828 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e8011590 con 0x7f94f813d120 2024-09-15T03:36:43.834 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7fb4d4a1e700 1 Processor -- start 2024-09-15T03:36:43.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7fb4d4a1e700 1 -- start start 2024-09-15T03:36:43.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7f94e67fc700 1 -- 172.21.15.110:0/453878828 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94e80116f0 con 0x7f94f813d120 2024-09-15T03:36:43.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7f94e67fc700 1 -- 172.21.15.110:0/453878828 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e8011ab0 con 0x7f94f813d120 2024-09-15T03:36:43.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7fee927fc700 1 --2- 172.21.15.110:0/3041231440 >> v2:172.21.15.177:3300/0 conn(0x7fee94139d90 0x7fee9413cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:43.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7f94fceea700 1 -- 172.21.15.110:0/453878828 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94f80ad920 con 0x7f94f813d120 2024-09-15T03:36:43.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7f94fceea700 1 -- 172.21.15.110:0/453878828 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94f813cb40 con 0x7f94f813d120 2024-09-15T03:36:43.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7f035530f700 1 -- 172.21.15.110:0/4138983868 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f0350068180 msgr2=0x7f0350141a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7f035530f700 1 --2- 172.21.15.110:0/4138983868 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f0350068180 0x7f0350141a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7f035530f700 1 -- 172.21.15.110:0/4138983868 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03500aced0 msgr2=0x7f035014d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7f035530f700 1 --2- 172.21.15.110:0/4138983868 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03500aced0 0x7f035014d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7f035530f700 1 -- 172.21.15.110:0/4138983868 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f034800bc30 con 0x7f035013d120 2024-09-15T03:36:43.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7fee92ffd700 1 --2- 172.21.15.110:0/3041231440 >> v2:172.21.15.177:3300/0 conn(0x7fee94139d90 0x7fee9413cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7fee92ffd700 1 -- 172.21.15.110:0/3041231440 >> v1:172.21.15.177:6789/0 conn(0x7fee9413a1f0 legacy=0x7fee9413d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7fee92ffd700 1 -- 172.21.15.110:0/3041231440 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee9406de30 con 0x7fee940678a0 2024-09-15T03:36:43.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7fee927fc700 1 --2- 172.21.15.110:0/3041231440 >> v2:172.21.15.177:3300/0 conn(0x7fee94139d90 0x7fee9413cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T03:36:43.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7f035530f700 1 --2- 172.21.15.110:0/4138983868 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f035013d120 0x7f0350150140 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f03500685a0 tx=0x7f0344008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:43.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7fee6627e700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fee600ad010 0x7fee600aac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:43.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7fee6627e700 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee600abcb0 con 0x7fee600ab300 2024-09-15T03:36:43.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7fee6627e700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee600a9cf0 con 0x7fee600acc40 2024-09-15T03:36:43.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7fee6627e700 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee601392c0 con 0x7fee600ad010 2024-09-15T03:36:43.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7fee65a7d700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fee600ad010 0x7fee600aac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:43.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7fee65a7d700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fee600ad010 0x7fee600aac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37262/0 (socket says 172.21.15.110:37262) 2024-09-15T03:36:43.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7fee6527c700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fee600acc40 0x7fee60138bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.110:59310/0 (socket says 172.21.15.110:59310) 2024-09-15T03:36:43.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7fee65a7d700 1 -- 172.21.15.110:0/357729423 learned_addr learned my addr 172.21.15.110:0/357729423 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:36:43.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.815+0000 7fb492ca4700 1 Processor -- start 2024-09-15T03:36:43.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7f94fceea700 1 -- 172.21.15.110:0/453878828 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f94f8064bd0 con 0x7f94f813d120 2024-09-15T03:36:43.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7fee92ffd700 1 --2- 172.21.15.110:0/3041231440 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee940678a0 0x7fee94139850 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fee940b2730 tx=0x7fee8800ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1504123b2881cd43 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:43.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7fee65a7d700 1 -- 172.21.15.110:0/357729423 >> v1:172.21.15.177:6789/0 conn(0x7fee600acc40 legacy=0x7fee60138bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7fee65a7d700 1 -- 172.21.15.110:0/357729423 >> v1:172.21.15.110:6789/0 conn(0x7fee600ab300 legacy=0x7fee6013eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:36:43.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.815+0000 7fb492ca4700 1 -- start start 2024-09-15T03:36:43.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.815+0000 7fb492ca4700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fb48c0ac7a0 0x7fb48c06bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:43.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7f034cff9700 1 -- 172.21.15.110:0/4138983868 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0344011600 con 0x7f035013d120 2024-09-15T03:36:43.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7fee65a7d700 1 -- 172.21.15.110:0/357729423 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee600ac5d0 con 0x7fee600ad010 2024-09-15T03:36:43.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.815+0000 7fb492ca4700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fb48c086800 0x7fb48c06c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:43.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.815+0000 7fb492ca4700 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb48c086db0 con 0x7fb48c0acb70 2024-09-15T03:36:43.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.815+0000 7fb492ca4700 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb48c06d1a0 con 0x7fb48c086800 2024-09-15T03:36:43.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.815+0000 7fb492ca4700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb48c06d480 con 0x7fb48c0ac7a0 2024-09-15T03:36:43.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.815+0000 7fb48bfff700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fb48c086800 0x7fb48c06c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:43.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.815+0000 7fb48bfff700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fb48c086800 0x7fb48c06c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37248/0 (socket says 172.21.15.110:37248) 2024-09-15T03:36:43.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.815+0000 7fb48bfff700 1 -- 172.21.15.110:0/3864889279 learned_addr learned my addr 172.21.15.110:0/3864889279 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:36:43.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7fb48b7fe700 1 -- 172.21.15.110:0/3864889279 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4089278189 0 0) 0x7fb48c086db0 con 0x7fb48c0acb70 2024-09-15T03:36:43.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7fb48b7fe700 1 -- 172.21.15.110:0/3864889279 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb474003550 con 0x7fb48c0acb70 2024-09-15T03:36:43.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7fb490a40700 1 --2- 172.21.15.110:0/3864889279 >> v2:172.21.15.177:3300/0 conn(0x7fb48c0ac7a0 0x7fb48c06bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:43.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7fb48bfff700 1 -- 172.21.15.110:0/3864889279 >> v2:172.21.15.177:3300/0 conn(0x7fb48c0ac7a0 msgr2=0x7fb48c06bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:36:43.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7fb48bfff700 1 --2- 172.21.15.110:0/3864889279 >> v2:172.21.15.177:3300/0 conn(0x7fb48c0ac7a0 0x7fb48c06bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7fb48bfff700 1 -- 172.21.15.110:0/3864889279 >> v1:172.21.15.110:6789/0 conn(0x7fb48c0acb70 legacy=0x7fb48c14db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7fb48bfff700 1 -- 172.21.15.110:0/3864889279 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb48c06c690 con 0x7fb48c086800 2024-09-15T03:36:43.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7fb490a40700 1 --2- 172.21.15.110:0/3864889279 >> v2:172.21.15.177:3300/0 conn(0x7fb48c0ac7a0 0x7fb48c06bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T03:36:43.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7f034cff9700 1 -- 172.21.15.110:0/4138983868 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0344011780 con 0x7f035013d120 2024-09-15T03:36:43.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7fb48bfff700 1 --2- 172.21.15.110:0/3864889279 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb48c086800 0x7fb48c06c150 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb48c0abce0 tx=0x7fb480009b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52ef4c29ed26ddcf server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:43.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7f034cff9700 1 -- 172.21.15.110:0/4138983868 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0344011b40 con 0x7f035013d120 2024-09-15T03:36:43.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7f0356d72700 1 -- 172.21.15.110:0/4138983868 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03400094f0 con 0x7f035013d120 2024-09-15T03:36:43.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7f0356d72700 1 -- 172.21.15.110:0/4138983868 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0350141490 con 0x7f035013d120 2024-09-15T03:36:43.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7fb48b7fe700 1 -- 172.21.15.110:0/3864889279 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4800092e0 con 0x7fb48c086800 2024-09-15T03:36:43.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7fb48b7fe700 1 -- 172.21.15.110:0/3864889279 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb480013070 con 0x7fb48c086800 2024-09-15T03:36:43.844 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.819+0000 7fb48b7fe700 1 -- 172.21.15.110:0/3864889279 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb480009440 con 0x7fb48c086800 2024-09-15T03:36:43.844 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fee65a7d700 1 --2- 172.21.15.110:0/357729423 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee600ad010 0x7fee600aac60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fee58009ed0 tx=0x7fee58009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96092df8912fd8c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:43.844 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7f94e67fc700 1 -- 172.21.15.110:0/453878828 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) v1 ==== 85847+0+0 (secure 0 0 0) 0x7f94e801b3b0 con 0x7f94f813d120 2024-09-15T03:36:43.844 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fa74a8a5700 1 Processor -- start 2024-09-15T03:36:43.845 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fa74a8a5700 1 -- start start 2024-09-15T03:36:43.845 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7f0356d72700 1 -- 172.21.15.110:0/4138983868 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0350064bd0 con 0x7f035013d120 2024-09-15T03:36:43.845 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fb492ca4700 1 -- 172.21.15.110:0/3864889279 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb48c086800 msgr2=0x7fb48c06c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.845 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fb492ca4700 1 --2- 172.21.15.110:0/3864889279 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb48c086800 0x7fb48c06c150 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb48c0abce0 tx=0x7fb480009b70 comp rx=0 tx=0).stop 2024-09-15T03:36:43.845 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fee91ffb700 1 -- 172.21.15.110:0/3041231440 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8800f070 con 0x7fee940678a0 2024-09-15T03:36:43.846 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fb4d4a1e700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fb4d0086900 0x7fb4d006db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:43.846 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7f94e67fc700 1 --2- 172.21.15.110:0/453878828 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f94d8067a80 0x7f94d8069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:43.846 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fee91ffb700 1 -- 172.21.15.110:0/3041231440 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee88008bf0 con 0x7fee940678a0 2024-09-15T03:36:43.846 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fee91ffb700 1 -- 172.21.15.110:0/3041231440 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8800a0d0 con 0x7fee940678a0 2024-09-15T03:36:43.847 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fb492ca4700 1 -- 172.21.15.110:0/3864889279 shutdown_connections 2024-09-15T03:36:43.847 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fb492ca4700 1 --2- 172.21.15.110:0/3864889279 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb48c086800 0x7fb48c06c150 secure :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb48c0abce0 tx=0x7fb480009b70 comp rx=0 tx=0).stop 2024-09-15T03:36:43.847 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fb492ca4700 1 --2- 172.21.15.110:0/3864889279 >> v2:172.21.15.177:3300/0 conn(0x7fb48c0ac7a0 0x7fb48c06bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.847 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fb492ca4700 1 -- 172.21.15.110:0/3864889279 >> 172.21.15.110:0/3864889279 conn(0x7fb48c067670 msgr2=0x7fb48c067a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:43.847 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7f94e67fc700 1 -- 172.21.15.110:0/453878828 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5262+0+0 (secure 0 0 0) 0x7f94e80860b0 con 0x7f94f813d120 2024-09-15T03:36:43.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fee57fff700 1 -- 172.21.15.110:0/357729423 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5800e070 con 0x7fee600ad010 2024-09-15T03:36:43.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fee57fff700 1 -- 172.21.15.110:0/357729423 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee5800ba00 con 0x7fee600ad010 2024-09-15T03:36:43.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fee57fff700 1 -- 172.21.15.110:0/357729423 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee580092e0 con 0x7fee600ad010 2024-09-15T03:36:43.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7f94e67fc700 1 --2- 172.21.15.110:0/453878828 >> [v2:172.21.15.177:6800/22423,v1:172.21.15.177:6801/22423] conn(0x7f94d806de50 0x7f94d8070280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:43.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7f94e67fc700 1 -- 172.21.15.110:0/453878828 --> [v2:172.21.15.177:6800/22423,v1:172.21.15.177:6801/22423] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f94d8070950 con 0x7f94d806de50 2024-09-15T03:36:43.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7f94e67fc700 1 -- 172.21.15.110:0/453878828 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_get_version_reply(handle=1 version=19) v2 ==== 24+0+0 (secure 0 0 0) 0x7f94e80863a0 con 0x7f94f813d120 2024-09-15T03:36:43.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7f94f659c700 1 --2- 172.21.15.110:0/453878828 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f94d8067a80 0x7f94d8069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:43.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fb492ca4700 1 -- 172.21.15.110:0/3864889279 shutdown_connections 2024-09-15T03:36:43.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fb492ca4700 1 -- 172.21.15.110:0/3864889279 wait complete. 2024-09-15T03:36:43.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fee9974a700 1 -- 172.21.15.110:0/3041231440 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee940678a0 msgr2=0x7fee94139850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fee9974a700 1 --2- 172.21.15.110:0/3041231440 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee940678a0 0x7fee94139850 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fee940b2730 tx=0x7fee8800ac80 comp rx=0 tx=0).stop 2024-09-15T03:36:43.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fee9974a700 1 -- 172.21.15.110:0/3041231440 shutdown_connections 2024-09-15T03:36:43.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fee9974a700 1 --2- 172.21.15.110:0/3041231440 >> v2:172.21.15.177:3300/0 conn(0x7fee94139d90 0x7fee9413cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fee9974a700 1 --2- 172.21.15.110:0/3041231440 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee940678a0 0x7fee94139850 secure :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fee940b2730 tx=0x7fee8800ac80 comp rx=0 tx=0).stop 2024-09-15T03:36:43.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fee9974a700 1 -- 172.21.15.110:0/3041231440 >> 172.21.15.110:0/3041231440 conn(0x7fee94069850 msgr2=0x7fee940b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:43.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fa74a8a5700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa74406a620 0x7fa7440acf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:43.851 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fa74a8a5700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fa74413d620 0x7fa744139bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:43.851 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fa74a8a5700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa74406c9e0 con 0x7fa74406c260 2024-09-15T03:36:43.851 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fa74a8a5700 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa74406c690 con 0x7fa74406a620 2024-09-15T03:36:43.851 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fa74a8a5700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa74406d910 con 0x7fa74413d620 2024-09-15T03:36:43.851 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fa7498a3700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fa74406a620 0x7fa7440acf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:43.851 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fa7490a2700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fa74406c260 0x7fa74413cf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.110:59318/0 (socket says 172.21.15.110:59318) 2024-09-15T03:36:43.851 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fa7490a2700 1 -- 172.21.15.110:0/283310630 learned_addr learned my addr 172.21.15.110:0/283310630 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-15T03:36:43.851 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fee9974a700 1 -- 172.21.15.110:0/3041231440 shutdown_connections 2024-09-15T03:36:43.852 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fa7498a3700 1 -- 172.21.15.110:0/283310630 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fa74413d620 msgr2=0x7fa744139bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:36:43.852 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fa7498a3700 1 --2- 172.21.15.110:0/283310630 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fa74413d620 0x7fa744139bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.852 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fa7498a3700 1 -- 172.21.15.110:0/283310630 >> v1:172.21.15.177:6789/0 conn(0x7fa74406c260 legacy=0x7fa74413cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.852 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fa7498a3700 1 -- 172.21.15.110:0/283310630 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7440ac630 con 0x7fa74406a620 2024-09-15T03:36:43.852 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fee9974a700 1 -- 172.21.15.110:0/3041231440 wait complete. 2024-09-15T03:36:43.852 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fa7498a3700 1 --2- 172.21.15.110:0/283310630 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa74406a620 0x7fa7440acf10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa7400098e0 tx=0x7fa7400091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d33d74440245d0d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:43.853 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fb4d4a1e700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fb4d00acaf0 0x7fb4d006bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:43.853 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fb4ce59c700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fb4d0086900 0x7fb4d006db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:43.853 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fb4cdd9b700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fb4d00acaf0 0x7fb4d006bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:43.853 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fb4cdd9b700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fb4d00acaf0 0x7fb4d006bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3301/0 says I am v2:172.21.15.110:44702/0 (socket says 172.21.15.110:44702) 2024-09-15T03:36:43.853 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fb4ce59c700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fb4d0086900 0x7fb4d006db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.110:49478/0 (socket says 172.21.15.110:49478) 2024-09-15T03:36:43.853 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fb4cdd9b700 1 -- 172.21.15.110:0/1821538855 learned_addr learned my addr 172.21.15.110:0/1821538855 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:36:43.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fb4d4a1e700 1 -- 172.21.15.110:0/1821538855 --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4d006d1a0 con 0x7fb4d0086310 2024-09-15T03:36:43.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fb4d4a1e700 1 -- 172.21.15.110:0/1821538855 --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4d006cfa0 con 0x7fb4d0086900 2024-09-15T03:36:43.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fb4d4a1e700 1 -- 172.21.15.110:0/1821538855 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d006d480 con 0x7fb4d00acaf0 2024-09-15T03:36:43.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fb4cdd9b700 1 -- 172.21.15.110:0/1821538855 >> v2:172.21.15.177:3300/0 conn(0x7fb4d0086900 msgr2=0x7fb4d006db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:36:43.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fb4cdd9b700 1 --2- 172.21.15.110:0/1821538855 >> v2:172.21.15.177:3300/0 conn(0x7fb4d0086900 0x7fb4d006db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fb4cdd9b700 1 -- 172.21.15.110:0/1821538855 >> v1:172.21.15.177:6789/0 conn(0x7fb4d0086310 legacy=0x7fb4d014dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fb4cdd9b700 1 -- 172.21.15.110:0/1821538855 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4d006b9d0 con 0x7fb4d00acaf0 2024-09-15T03:36:43.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fee6627e700 1 -- 172.21.15.110:0/357729423 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee600ad010 msgr2=0x7fee600aac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fee6627e700 1 --2- 172.21.15.110:0/357729423 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee600ad010 0x7fee600aac60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fee58009ed0 tx=0x7fee58009840 comp rx=0 tx=0).stop 2024-09-15T03:36:43.855 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fb492ca4700 1 Processor -- start 2024-09-15T03:36:43.855 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fa7488a1700 1 -- 172.21.15.110:0/283310630 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa740009e80 con 0x7fa74406a620 2024-09-15T03:36:43.855 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7f94f659c700 1 --2- 172.21.15.110:0/453878828 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f94d8067a80 0x7f94d8069f30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f94ec003a30 tx=0x7f94ec007ec0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:43.855 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7f94f5d9b700 1 --2- 172.21.15.110:0/453878828 >> [v2:172.21.15.177:6800/22423,v1:172.21.15.177:6801/22423] conn(0x7f94d806de50 0x7f94d8070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:43.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fa7488a1700 1 -- 172.21.15.110:0/283310630 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa74000b860 con 0x7fa74406a620 2024-09-15T03:36:43.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fa7488a1700 1 -- 172.21.15.110:0/283310630 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa74000bbe0 con 0x7fa74406a620 2024-09-15T03:36:43.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7f034cff9700 1 -- 172.21.15.110:0/4138983868 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 5) v1 ==== 85847+0+0 (secure 0 0 0) 0x7f0344039400 con 0x7f035013d120 2024-09-15T03:36:43.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fb4cdd9b700 1 --2- 172.21.15.110:0/1821538855 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fb4d00acaf0 0x7fb4d006bef0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb4c8014ac0 tx=0x7fb4c80143c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3650b79301794e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:43.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fb4cd59a700 1 -- 172.21.15.110:0/1821538855 <== mon.2 v2:172.21.15.110:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c8013bd0 con 0x7fb4d00acaf0 2024-09-15T03:36:43.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fee6627e700 1 -- 172.21.15.110:0/357729423 shutdown_connections 2024-09-15T03:36:43.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fee6627e700 1 --2- 172.21.15.110:0/357729423 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee600ad010 0x7fee600aac60 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fee6627e700 1 -- 172.21.15.110:0/357729423 >> 172.21.15.110:0/357729423 conn(0x7fee60067670 msgr2=0x7fee60067a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:43.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fb4cd59a700 1 -- 172.21.15.110:0/1821538855 <== mon.2 v2:172.21.15.110:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4c8013d30 con 0x7fb4d00acaf0 2024-09-15T03:36:43.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fb4cd59a700 1 -- 172.21.15.110:0/1821538855 <== mon.2 v2:172.21.15.110:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c80140f0 con 0x7fb4d00acaf0 2024-09-15T03:36:43.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fee6627e700 1 -- 172.21.15.110:0/357729423 shutdown_connections 2024-09-15T03:36:43.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7f034cff9700 1 --2- 172.21.15.110:0/4138983868 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f033c067bd0 0x7f033c06a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:43.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fee9974a700 1 Processor -- start 2024-09-15T03:36:43.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.823+0000 7fb492ca4700 1 -- start start 2024-09-15T03:36:43.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fb492ca4700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb48c0ac7a0 0x7fb48c079980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:43.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fb492ca4700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fb48c0acb70 0x7fb48c079ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:43.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fb492ca4700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb48c159e20 0x7fb48c15a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:43.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fb492ca4700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c07a400 con 0x7fb48c0ac7a0 2024-09-15T03:36:43.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fb492ca4700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c076760 con 0x7fb48c159e20 2024-09-15T03:36:43.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fb492ca4700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c078540 con 0x7fb48c0acb70 2024-09-15T03:36:43.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fb48bfff700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fb48c0acb70 0x7fb48c079ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:43.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fb48bfff700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fb48c0acb70 0x7fb48c079ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3301/0 says I am v2:172.21.15.110:44716/0 (socket says 172.21.15.110:44716) 2024-09-15T03:36:43.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fb48bfff700 1 -- 172.21.15.110:0/1374622977 learned_addr learned my addr 172.21.15.110:0/1374622977 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:36:43.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7f034cff9700 1 -- 172.21.15.110:0/4138983868 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5262+0+0 (secure 0 0 0) 0x7f0344085c50 con 0x7f035013d120 2024-09-15T03:36:43.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7f034cff9700 1 --2- 172.21.15.110:0/4138983868 >> [v2:172.21.15.110:6808/22354,v1:172.21.15.110:6809/22354] conn(0x7f033c06dfa0 0x7f033c0703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:43.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7f034cff9700 1 -- 172.21.15.110:0/4138983868 --> [v2:172.21.15.110:6808/22354,v1:172.21.15.110:6809/22354] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f033c070aa0 con 0x7f033c06dfa0 2024-09-15T03:36:43.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7f034cff9700 1 -- 172.21.15.110:0/4138983868 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_get_version_reply(handle=1 version=19) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0344085f80 con 0x7f035013d120 2024-09-15T03:36:43.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7f034ffff700 1 --2- 172.21.15.110:0/4138983868 >> [v2:172.21.15.110:6808/22354,v1:172.21.15.110:6809/22354] conn(0x7f033c06dfa0 0x7f033c0703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:43.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fa74a8a5700 1 -- 172.21.15.110:0/283310630 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa74406a620 msgr2=0x7fa7440acf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fa74a8a5700 1 --2- 172.21.15.110:0/283310630 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa74406a620 0x7fa7440acf10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa7400098e0 tx=0x7fa7400091a0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fa74a8a5700 1 -- 172.21.15.110:0/283310630 shutdown_connections 2024-09-15T03:36:43.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fa74a8a5700 1 --2- 172.21.15.110:0/283310630 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fa74413d620 0x7fa744139bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fa74a8a5700 1 --2- 172.21.15.110:0/283310630 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa74406a620 0x7fa7440acf10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fa74a8a5700 1 -- 172.21.15.110:0/283310630 >> 172.21.15.110:0/283310630 conn(0x7fa744067810 msgr2=0x7fa744067c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:43.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fee9974a700 1 -- start start 2024-09-15T03:36:43.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7f7c362bd700 1 Processor -- start 2024-09-15T03:36:43.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7f94f5d9b700 1 --2- 172.21.15.110:0/453878828 >> [v2:172.21.15.177:6800/22423,v1:172.21.15.177:6801/22423] conn(0x7f94d806de50 0x7f94d8070280 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:43.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7f7c362bd700 1 -- start start 2024-09-15T03:36:43.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fb491241700 1 --2- 172.21.15.110:0/1374622977 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb48c159e20 0x7fb48c15a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:43.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7f0354b0e700 1 --2- 172.21.15.110:0/4138983868 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f033c067bd0 0x7f033c06a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:43.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fee6627e700 1 -- 172.21.15.110:0/357729423 wait complete. 2024-09-15T03:36:43.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fb48bfff700 1 -- 172.21.15.110:0/1374622977 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb48c159e20 msgr2=0x7fb48c15a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fb48bfff700 1 --2- 172.21.15.110:0/1374622977 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb48c159e20 0x7fb48c15a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fb48bfff700 1 -- 172.21.15.110:0/1374622977 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb48c0ac7a0 msgr2=0x7fb48c079980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T03:36:43.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fb48bfff700 1 --2- 172.21.15.110:0/1374622977 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb48c0ac7a0 0x7fb48c079980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fb48bfff700 1 -- 172.21.15.110:0/1374622977 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb480009e10 con 0x7fb48c0acb70 2024-09-15T03:36:43.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7f034ffff700 1 --2- 172.21.15.110:0/4138983868 >> [v2:172.21.15.110:6808/22354,v1:172.21.15.110:6809/22354] conn(0x7f033c06dfa0 0x7f033c0703d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:43.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fee9974a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee94139d90 0x7fee940ad120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:43.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fee9974a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fee9413a1f0 0x7fee940af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:43.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fee9974a700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fee940aef20 0x7fee94154a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:43.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fee9974a700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee940aea90 con 0x7fee94139d90 2024-09-15T03:36:43.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fee9974a700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee940af390 con 0x7fee9413a1f0 2024-09-15T03:36:43.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fee9974a700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee9413d640 con 0x7fee940aef20 2024-09-15T03:36:43.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fee92ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee94139d90 0x7fee940ad120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:43.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fee92ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee94139d90 0x7fee940ad120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37276/0 (socket says 172.21.15.110:37276) 2024-09-15T03:36:43.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fee92ffd700 1 -- 172.21.15.110:0/1100349915 learned_addr learned my addr 172.21.15.110:0/1100349915 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:36:43.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fee93fff700 1 --2- 172.21.15.110:0/1100349915 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fee940aef20 0x7fee94154a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:43.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fee927fc700 1 --2- 172.21.15.110:0/1100349915 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fee9413a1f0 0x7fee940af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:43.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fb48bfff700 1 --2- 172.21.15.110:0/1374622977 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fb48c0acb70 0x7fb48c079ec0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb480009bb0 tx=0x7fb480013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:43.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fee6627e700 1 Processor -- start 2024-09-15T03:36:43.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fee6627e700 1 -- start start 2024-09-15T03:36:43.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fee93fff700 1 -- 172.21.15.110:0/1100349915 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fee9413a1f0 msgr2=0x7fee940af8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fee93fff700 1 --2- 172.21.15.110:0/1100349915 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fee9413a1f0 0x7fee940af8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fee93fff700 1 -- 172.21.15.110:0/1100349915 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee94139d90 msgr2=0x7fee940ad120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fee93fff700 1 --2- 172.21.15.110:0/1100349915 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee94139d90 0x7fee940ad120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fee93fff700 1 -- 172.21.15.110:0/1100349915 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee88007040 con 0x7fee940aef20 2024-09-15T03:36:43.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fb488ff9700 1 -- 172.21.15.110:0/1374622977 <== mon.2 v2:172.21.15.110:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48000b560 con 0x7fb48c0acb70 2024-09-15T03:36:43.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fb488ff9700 1 -- 172.21.15.110:0/1374622977 <== mon.2 v2:172.21.15.110:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb48001c070 con 0x7fb48c0acb70 2024-09-15T03:36:43.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fb488ff9700 1 -- 172.21.15.110:0/1374622977 <== mon.2 v2:172.21.15.110:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48000b910 con 0x7fb48c0acb70 2024-09-15T03:36:43.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fa74a8a5700 1 -- 172.21.15.110:0/283310630 shutdown_connections 2024-09-15T03:36:43.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7f94e67fc700 1 -- 172.21.15.110:0/453878828 <== osd.7 v2:172.21.15.177:6800/22423 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f94d8070950 con 0x7f94d806de50 2024-09-15T03:36:43.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7f0354b0e700 1 --2- 172.21.15.110:0/4138983868 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f033c067bd0 0x7f033c06a080 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f034000c010 tx=0x7f0340017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:43.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fb492ca4700 1 -- 172.21.15.110:0/1374622977 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb48c15a800 con 0x7fb48c0acb70 2024-09-15T03:36:43.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fb492ca4700 1 -- 172.21.15.110:0/1374622977 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb48c15cfd0 con 0x7fb48c0acb70 2024-09-15T03:36:43.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7f034cff9700 1 -- 172.21.15.110:0/4138983868 <== osd.1 v2:172.21.15.110:6808/22354 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f033c070aa0 con 0x7f033c06dfa0 2024-09-15T03:36:43.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fee93fff700 1 --2- 172.21.15.110:0/1100349915 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fee940aef20 0x7fee94154a30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fee9406ca40 tx=0x7fee8c0015f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:43.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fee6627e700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee600ab300 0x7fee6014e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:43.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fee6627e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fee600acc40 0x7fee600a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:43.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fee6627e700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fee600ad010 0x7fee600a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:43.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fee6627e700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee600a8450 con 0x7fee600ab300 2024-09-15T03:36:43.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fee6627e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee600a7560 con 0x7fee600acc40 2024-09-15T03:36:43.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fee6627e700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee600a56b0 con 0x7fee600ad010 2024-09-15T03:36:43.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fee6527c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee600ab300 0x7fee6014e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:43.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fee6527c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee600ab300 0x7fee6014e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37290/0 (socket says 172.21.15.110:37290) 2024-09-15T03:36:43.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fee6527c700 1 -- 172.21.15.110:0/1412490983 learned_addr learned my addr 172.21.15.110:0/1412490983 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:36:43.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fa74a8a5700 1 -- 172.21.15.110:0/283310630 wait complete. 2024-09-15T03:36:43.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fee64a7b700 1 --2- 172.21.15.110:0/1412490983 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fee600acc40 0x7fee600a7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:43.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fee6527c700 1 -- 172.21.15.110:0/1412490983 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fee600ad010 msgr2=0x7fee600a5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fee6527c700 1 --2- 172.21.15.110:0/1412490983 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fee600ad010 0x7fee600a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fee6527c700 1 -- 172.21.15.110:0/1412490983 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fee600acc40 msgr2=0x7fee600a7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fee6527c700 1 --2- 172.21.15.110:0/1412490983 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fee600acc40 0x7fee600a7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fee6527c700 1 -- 172.21.15.110:0/1412490983 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee58009ae0 con 0x7fee600ab300 2024-09-15T03:36:43.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fb4d4a1e700 1 -- 172.21.15.110:0/1821538855 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fb4d00acaf0 msgr2=0x7fb4d006bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fb4d4a1e700 1 --2- 172.21.15.110:0/1821538855 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fb4d00acaf0 0x7fb4d006bef0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb4c8014ac0 tx=0x7fb4c80143c0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fb4d4a1e700 1 -- 172.21.15.110:0/1821538855 shutdown_connections 2024-09-15T03:36:43.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fb4d4a1e700 1 --2- 172.21.15.110:0/1821538855 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fb4d00acaf0 0x7fb4d006bef0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fb4d4a1e700 1 --2- 172.21.15.110:0/1821538855 >> v2:172.21.15.177:3300/0 conn(0x7fb4d0086900 0x7fb4d006db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fb4d4a1e700 1 -- 172.21.15.110:0/1821538855 >> 172.21.15.110:0/1821538855 conn(0x7fb4d0067670 msgr2=0x7fb4d0067a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:43.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7f7c362bd700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f7c300acd30 0x7f7c3006e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:43.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7f7c362bd700 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c3013ccd0 con 0x7f7c300ad100 2024-09-15T03:36:43.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7f7c362bd700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c300ac650 con 0x7f7c30139ad0 2024-09-15T03:36:43.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7f7c362bd700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3013a680 con 0x7f7c300acd30 2024-09-15T03:36:43.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7f7c352bb700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f7c300acd30 0x7f7c3006e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:43.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7f7c352bb700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f7c300acd30 0x7f7c3006e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3301/0 says I am v2:172.21.15.110:44742/0 (socket says 172.21.15.110:44742) 2024-09-15T03:36:43.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7f7c352bb700 1 -- 172.21.15.110:0/908176074 learned_addr learned my addr 172.21.15.110:0/908176074 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:36:43.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7f7c27fff700 1 -- 172.21.15.110:0/908176074 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4229507616 0 0) 0x7f7c3013ccd0 con 0x7f7c300ad100 2024-09-15T03:36:43.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7f7c27fff700 1 -- 172.21.15.110:0/908176074 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c1c003570 con 0x7f7c300ad100 2024-09-15T03:36:43.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7f7c352bb700 1 -- 172.21.15.110:0/908176074 >> v1:172.21.15.177:6789/0 conn(0x7f7c30139ad0 legacy=0x7f7c30139f70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:36:43.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7f7c352bb700 1 -- 172.21.15.110:0/908176074 >> v1:172.21.15.110:6789/0 conn(0x7f7c300ad100 legacy=0x7f7c30136b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7f7c352bb700 1 -- 172.21.15.110:0/908176074 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c3013db50 con 0x7f7c300acd30 2024-09-15T03:36:43.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fee7affd700 1 -- 172.21.15.110:0/1100349915 <== mon.2 v2:172.21.15.110:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8c012630 con 0x7fee940aef20 2024-09-15T03:36:43.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fee6527c700 1 --2- 172.21.15.110:0/1412490983 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee600ab300 0x7fee6014e250 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fee5c00c7c0 tx=0x7fee5c007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:43.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fee7affd700 1 -- 172.21.15.110:0/1100349915 <== mon.2 v2:172.21.15.110:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee8c0127d0 con 0x7fee940aef20 2024-09-15T03:36:43.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fee7affd700 1 -- 172.21.15.110:0/1100349915 <== mon.2 v2:172.21.15.110:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8c012b90 con 0x7fee940aef20 2024-09-15T03:36:43.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.827+0000 7fee9974a700 1 -- 172.21.15.110:0/1100349915 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee940ad890 con 0x7fee940aef20 2024-09-15T03:36:43.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.831+0000 7fee9974a700 1 -- 172.21.15.110:0/1100349915 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee940b0010 con 0x7fee940aef20 2024-09-15T03:36:43.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.831+0000 7fa74a8a5700 1 Processor -- start 2024-09-15T03:36:43.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.831+0000 7fb4d4a1e700 1 -- 172.21.15.110:0/1821538855 shutdown_connections 2024-09-15T03:36:43.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.831+0000 7f7c352bb700 1 --2- 172.21.15.110:0/908176074 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f7c300acd30 0x7f7c3006e860 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7c28009f50 tx=0x7f7c28009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a41cfaa144a38b11 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:43.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.831+0000 7fb4d4a1e700 1 -- 172.21.15.110:0/1821538855 wait complete. 2024-09-15T03:36:43.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.831+0000 7fee557fa700 1 -- 172.21.15.110:0/1412490983 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5c007300 con 0x7fee600ab300 2024-09-15T03:36:43.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.831+0000 7fee9974a700 1 -- 172.21.15.110:0/1100349915 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fee74000f20 con 0x7fee940aef20 2024-09-15T03:36:43.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.831+0000 7fee6627e700 1 -- 172.21.15.110:0/1412490983 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee6014ecd0 con 0x7fee600ab300 2024-09-15T03:36:43.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.831+0000 7fee6627e700 1 -- 172.21.15.110:0/1412490983 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee600a8a80 con 0x7fee600ab300 2024-09-15T03:36:43.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.831+0000 7fee557fa700 1 -- 172.21.15.110:0/1412490983 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee5c007480 con 0x7fee600ab300 2024-09-15T03:36:43.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.831+0000 7fee557fa700 1 -- 172.21.15.110:0/1412490983 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5c004830 con 0x7fee600ab300 2024-09-15T03:36:43.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.831+0000 7fb492ca4700 1 -- 172.21.15.110:0/1374622977 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb48c159a40 con 0x7fb48c0acb70 2024-09-15T03:36:43.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.831+0000 7fb488ff9700 1 -- 172.21.15.110:0/1374622977 <== mon.2 v2:172.21.15.110:3301/0 4 ==== mgrmap(e 5) v1 ==== 85847+0+0 (secure 0 0 0) 0x7fb48000bd00 con 0x7fb48c0acb70 2024-09-15T03:36:43.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.831+0000 7fb488ff9700 1 --2- 172.21.15.110:0/1374622977 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fb478067b30 0x7fb478069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:43.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.831+0000 7fb488ff9700 1 -- 172.21.15.110:0/1374622977 <== mon.2 v2:172.21.15.110:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5262+0+0 (secure 0 0 0) 0x7fb48001e070 con 0x7fb48c0acb70 2024-09-15T03:36:43.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.831+0000 7fb488ff9700 1 --2- 172.21.15.110:0/1374622977 >> [v2:172.21.15.110:6800/22352,v1:172.21.15.110:6801/22352] conn(0x7fb47806df00 0x7fb478070330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:43.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.831+0000 7fb490a40700 1 --2- 172.21.15.110:0/1374622977 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fb478067b30 0x7fb478069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:43.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.831+0000 7fb491241700 1 --2- 172.21.15.110:0/1374622977 >> [v2:172.21.15.110:6800/22352,v1:172.21.15.110:6801/22352] conn(0x7fb47806df00 0x7fb478070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:43.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.831+0000 7fb488ff9700 1 -- 172.21.15.110:0/1374622977 --> [v2:172.21.15.110:6800/22352,v1:172.21.15.110:6801/22352] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb478070a00 con 0x7fb47806df00 2024-09-15T03:36:43.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.831+0000 7fb488ff9700 1 -- 172.21.15.110:0/1374622977 <== mon.2 v2:172.21.15.110:3301/0 6 ==== mon_get_version_reply(handle=1 version=19) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb480052ed0 con 0x7fb48c0acb70 2024-09-15T03:36:43.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.831+0000 7f7c27fff700 1 -- 172.21.15.110:0/908176074 <== mon.2 v2:172.21.15.110:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2800e070 con 0x7f7c300acd30 2024-09-15T03:36:43.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.831+0000 7f7c27fff700 1 -- 172.21.15.110:0/908176074 <== mon.2 v2:172.21.15.110:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c2800b9c0 con 0x7f7c300acd30 2024-09-15T03:36:43.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.831+0000 7f7c27fff700 1 -- 172.21.15.110:0/908176074 <== mon.2 v2:172.21.15.110:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c280092e0 con 0x7f7c300acd30 2024-09-15T03:36:43.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.831+0000 7fa74a8a5700 1 -- start start 2024-09-15T03:36:43.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.831+0000 7fee6627e700 1 -- 172.21.15.110:0/1412490983 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fee40000f20 con 0x7fee600ab300 2024-09-15T03:36:43.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.831+0000 7fb4d4a1e700 1 Processor -- start 2024-09-15T03:36:43.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.831+0000 7fb4d4a1e700 1 -- start start 2024-09-15T03:36:43.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.831+0000 7fee557fa700 1 -- 172.21.15.110:0/1412490983 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) v1 ==== 85847+0+0 (secure 0 0 0) 0x7fee5c004c20 con 0x7fee600ab300 2024-09-15T03:36:43.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.831+0000 7fee7affd700 1 -- 172.21.15.110:0/1100349915 <== mon.2 v2:172.21.15.110:3301/0 4 ==== mgrmap(e 5) v1 ==== 85847+0+0 (secure 0 0 0) 0x7fee8c024430 con 0x7fee940aef20 2024-09-15T03:36:43.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.835+0000 7fa74a8a5700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa74406a620 0x7fa7441421f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:43.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.831+0000 7fb491241700 1 --2- 172.21.15.110:0/1374622977 >> [v2:172.21.15.110:6800/22352,v1:172.21.15.110:6801/22352] conn(0x7fb47806df00 0x7fb478070330 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:43.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.831+0000 7fb490a40700 1 --2- 172.21.15.110:0/1374622977 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fb478067b30 0x7fb478069fe0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb47c009500 tx=0x7fb47c0091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:43.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.835+0000 7fa74a8a5700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fa74406c260 0x7fa7440a8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:43.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.835+0000 7fa74a8a5700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa74413d620 0x7fa744140480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:43.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.835+0000 7fa74a8a5700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa744141580 con 0x7fa74406a620 2024-09-15T03:36:43.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.835+0000 7fa74a8a5700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa74413fff0 con 0x7fa74413d620 2024-09-15T03:36:43.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.835+0000 7fa74a8a5700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa744141760 con 0x7fa74406c260 2024-09-15T03:36:43.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.835+0000 7fa7490a2700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fa74406c260 0x7fa7440a8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:43.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.835+0000 7fa7490a2700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fa74406c260 0x7fa7440a8ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3301/0 says I am v2:172.21.15.110:44764/0 (socket says 172.21.15.110:44764) 2024-09-15T03:36:43.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.835+0000 7fa7490a2700 1 -- 172.21.15.110:0/1230917431 learned_addr learned my addr 172.21.15.110:0/1230917431 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:36:43.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.835+0000 7fb4d4a1e700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4d0086310 0x7fb4d00a86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:43.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.835+0000 7fa74a0a4700 1 --2- 172.21.15.110:0/1230917431 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa74413d620 0x7fa744140480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:43.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.835+0000 7fee7affd700 1 --2- 172.21.15.110:0/1100349915 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fee80067bd0 0x7fee8006a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:43.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.835+0000 7fee7affd700 1 -- 172.21.15.110:0/1100349915 <== mon.2 v2:172.21.15.110:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5262+0+0 (secure 0 0 0) 0x7fee8c085d40 con 0x7fee940aef20 2024-09-15T03:36:43.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.835+0000 7fb4d4a1e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4d0086900 0x7fb4d00a8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:43.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.835+0000 7fb4d4a1e700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fb4d00acaf0 0x7fb4d0159cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:43.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.835+0000 7fb4d4a1e700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d0076ce0 con 0x7fb4d0086310 2024-09-15T03:36:43.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.835+0000 7fb4d4a1e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d00a5650 con 0x7fb4d0086900 2024-09-15T03:36:43.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.835+0000 7fb4d4a1e700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d00a93c0 con 0x7fb4d00acaf0 2024-09-15T03:36:43.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.835+0000 7fee92ffd700 1 --2- 172.21.15.110:0/1100349915 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fee80067bd0 0x7fee8006a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:43.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.835+0000 7fee7affd700 1 --2- 172.21.15.110:0/1100349915 >> [v2:172.21.15.177:6822/22425,v1:172.21.15.177:6824/22425] conn(0x7fee8006dfa0 0x7fee800703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:43.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.835+0000 7fee7affd700 1 -- 172.21.15.110:0/1100349915 --> [v2:172.21.15.177:6822/22425,v1:172.21.15.177:6824/22425] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fee80070aa0 con 0x7fee8006dfa0 2024-09-15T03:36:43.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.835+0000 7fee7affd700 1 -- 172.21.15.110:0/1100349915 <== mon.2 v2:172.21.15.110:3301/0 6 ==== mon_get_version_reply(handle=1 version=19) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee8c086070 con 0x7fee940aef20 2024-09-15T03:36:43.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.835+0000 7fb488ff9700 1 -- 172.21.15.110:0/1374622977 <== osd.0 v2:172.21.15.110:6800/22352 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fb478070a00 con 0x7fb47806df00 2024-09-15T03:36:43.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.835+0000 7fb4ce59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4d0086310 0x7fb4d00a86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:43.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.835+0000 7fb4ce59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4d0086310 0x7fb4d00a86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37306/0 (socket says 172.21.15.110:37306) 2024-09-15T03:36:43.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.835+0000 7fb4ce59c700 1 -- 172.21.15.110:0/1996833262 learned_addr learned my addr 172.21.15.110:0/1996833262 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:36:43.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.835+0000 7fb4ced9d700 1 --2- 172.21.15.110:0/1996833262 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fb4d00acaf0 0x7fb4d0159cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:43.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.835+0000 7fb4cdd9b700 1 --2- 172.21.15.110:0/1996833262 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4d0086900 0x7fb4d00a8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:43.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.835+0000 7fa7490a2700 1 -- 172.21.15.110:0/1230917431 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa74413d620 msgr2=0x7fa744140480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.835+0000 7fa7490a2700 1 --2- 172.21.15.110:0/1230917431 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa74413d620 0x7fa744140480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.835+0000 7fa7490a2700 1 -- 172.21.15.110:0/1230917431 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa74406a620 msgr2=0x7fa7441421f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T03:36:43.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.835+0000 7fa7490a2700 1 --2- 172.21.15.110:0/1230917431 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa74406a620 0x7fa7441421f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.835+0000 7fa7490a2700 1 -- 172.21.15.110:0/1230917431 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa740009420 con 0x7fa74406c260 2024-09-15T03:36:43.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.835+0000 7fb4ce59c700 1 -- 172.21.15.110:0/1996833262 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fb4d00acaf0 msgr2=0x7fb4d0159cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.835+0000 7fb4ce59c700 1 --2- 172.21.15.110:0/1996833262 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fb4d00acaf0 0x7fb4d0159cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.835+0000 7fb4ce59c700 1 -- 172.21.15.110:0/1996833262 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4d0086900 msgr2=0x7fb4d00a8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.835+0000 7fb4ce59c700 1 --2- 172.21.15.110:0/1996833262 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4d0086900 0x7fb4d00a8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.835+0000 7fb4ce59c700 1 -- 172.21.15.110:0/1996833262 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4c8014600 con 0x7fb4d0086310 2024-09-15T03:36:43.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.835+0000 7fa7490a2700 1 --2- 172.21.15.110:0/1230917431 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fa74406c260 0x7fa7440a8ab0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa73400c7c0 tx=0x7fa734007f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:43.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.835+0000 7fee927fc700 1 --2- 172.21.15.110:0/1100349915 >> [v2:172.21.15.177:6822/22425,v1:172.21.15.177:6824/22425] conn(0x7fee8006dfa0 0x7fee800703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:43.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.835+0000 7fee92ffd700 1 --2- 172.21.15.110:0/1100349915 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fee80067bd0 0x7fee8006a080 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fee8800aed0 tx=0x7fee88019000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:43.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.835+0000 7fb4ce59c700 1 --2- 172.21.15.110:0/1996833262 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4d0086310 0x7fb4d00a86b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb4c000d960 tx=0x7fb4c0009450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:43.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.835+0000 7fa739ffb700 1 -- 172.21.15.110:0/1230917431 <== mon.2 v2:172.21.15.110:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa734005b30 con 0x7fa74406c260 2024-09-15T03:36:43.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.835+0000 7fa739ffb700 1 -- 172.21.15.110:0/1230917431 <== mon.2 v2:172.21.15.110:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa734005cb0 con 0x7fa74406c260 2024-09-15T03:36:43.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.835+0000 7fa74a8a5700 1 -- 172.21.15.110:0/1230917431 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa744141a40 con 0x7fa74406c260 2024-09-15T03:36:43.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.835+0000 7fa74a8a5700 1 -- 172.21.15.110:0/1230917431 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa74413fb20 con 0x7fa74406c260 2024-09-15T03:36:43.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.835+0000 7fee927fc700 1 --2- 172.21.15.110:0/1100349915 >> [v2:172.21.15.177:6822/22425,v1:172.21.15.177:6824/22425] conn(0x7fee8006dfa0 0x7fee800703d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:43.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.835+0000 7fb4be7fc700 1 -- 172.21.15.110:0/1996833262 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c000c040 con 0x7fb4d0086310 2024-09-15T03:36:43.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.835+0000 7fb4be7fc700 1 -- 172.21.15.110:0/1996833262 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4c0006ad0 con 0x7fb4d0086310 2024-09-15T03:36:43.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.835+0000 7fb4be7fc700 1 -- 172.21.15.110:0/1996833262 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c0006e70 con 0x7fb4d0086310 2024-09-15T03:36:43.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.835+0000 7fb4d4a1e700 1 -- 172.21.15.110:0/1996833262 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4d00a8220 con 0x7fb4d0086310 2024-09-15T03:36:43.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.835+0000 7fb4d4a1e700 1 -- 172.21.15.110:0/1996833262 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4d015a450 con 0x7fb4d0086310 2024-09-15T03:36:43.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.835+0000 7fee557fa700 1 --2- 172.21.15.110:0/1412490983 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fee44067a80 0x7fee44069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:43.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.835+0000 7fee557fa700 1 -- 172.21.15.110:0/1412490983 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5262+0+0 (secure 0 0 0) 0x7fee5c084a00 con 0x7fee600ab300 2024-09-15T03:36:43.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.835+0000 7fee557fa700 1 --2- 172.21.15.110:0/1412490983 >> [v2:172.21.15.110:6822/22356,v1:172.21.15.110:6824/22356] conn(0x7fee4406de50 0x7fee44070280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:43.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.835+0000 7fee557fa700 1 -- 172.21.15.110:0/1412490983 --> [v2:172.21.15.110:6822/22356,v1:172.21.15.110:6824/22356] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fee44070950 con 0x7fee4406de50 2024-09-15T03:36:43.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.835+0000 7fee557fa700 1 -- 172.21.15.110:0/1412490983 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_get_version_reply(handle=1 version=19) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee5c089240 con 0x7fee600ab300 2024-09-15T03:36:43.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.835+0000 7fee65a7d700 1 --2- 172.21.15.110:0/1412490983 >> [v2:172.21.15.110:6822/22356,v1:172.21.15.110:6824/22356] conn(0x7fee4406de50 0x7fee44070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:43.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.835+0000 7fee64a7b700 1 --2- 172.21.15.110:0/1412490983 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fee44067a80 0x7fee44069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:43.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.839+0000 7fb4be7fc700 1 -- 172.21.15.110:0/1996833262 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) v1 ==== 85847+0+0 (secure 0 0 0) 0x7fb4c000b9e0 con 0x7fb4d0086310 2024-09-15T03:36:43.892 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.839+0000 7fee65a7d700 1 --2- 172.21.15.110:0/1412490983 >> [v2:172.21.15.110:6822/22356,v1:172.21.15.110:6824/22356] conn(0x7fee4406de50 0x7fee44070280 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:43.892 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.839+0000 7fee7affd700 1 -- 172.21.15.110:0/1100349915 <== osd.4 v2:172.21.15.177:6822/22425 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fee80070aa0 con 0x7fee8006dfa0 2024-09-15T03:36:43.892 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.839+0000 7fb4be7fc700 1 --2- 172.21.15.110:0/1996833262 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fb4b0067a80 0x7fb4b0069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:43.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.839+0000 7fee557fa700 1 -- 172.21.15.110:0/1412490983 <== osd.2 v2:172.21.15.110:6822/22356 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fee44070950 con 0x7fee4406de50 2024-09-15T03:36:43.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.839+0000 7fb4be7fc700 1 -- 172.21.15.110:0/1996833262 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5262+0+0 (secure 0 0 0) 0x7fb4c0085db0 con 0x7fb4d0086310 2024-09-15T03:36:43.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.839+0000 7fb4cdd9b700 1 --2- 172.21.15.110:0/1996833262 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fb4b0067a80 0x7fb4b0069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:43.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.839+0000 7fb4cdd9b700 1 --2- 172.21.15.110:0/1996833262 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fb4b0067a80 0x7fb4b0069f30 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb4c80145d0 tx=0x7fb4c801d000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:43.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.839+0000 7fa739ffb700 1 -- 172.21.15.110:0/1230917431 <== mon.2 v2:172.21.15.110:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa734004a80 con 0x7fa74406c260 2024-09-15T03:36:43.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.839+0000 7fa739ffb700 1 -- 172.21.15.110:0/1230917431 <== mon.2 v2:172.21.15.110:3301/0 4 ==== mgrmap(e 5) v1 ==== 85847+0+0 (secure 0 0 0) 0x7fa7340070d0 con 0x7fa74406c260 2024-09-15T03:36:43.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.839+0000 7fa739ffb700 1 --2- 172.21.15.110:0/1230917431 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fa728067bd0 0x7fa72806a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:43.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.839+0000 7fa739ffb700 1 -- 172.21.15.110:0/1230917431 <== mon.2 v2:172.21.15.110:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5262+0+0 (secure 0 0 0) 0x7fa7340848a0 con 0x7fa74406c260 2024-09-15T03:36:43.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.839+0000 7fa74a8a5700 1 --2- 172.21.15.110:0/1230917431 >> [v2:172.21.15.110:6816/22355,v1:172.21.15.110:6817/22355] conn(0x7fa7240015f0 0x7fa724003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:43.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.839+0000 7fa74a8a5700 1 -- 172.21.15.110:0/1230917431 --> [v2:172.21.15.110:6816/22355,v1:172.21.15.110:6817/22355] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa724006ae0 con 0x7fa7240015f0 2024-09-15T03:36:43.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.839+0000 7fee64a7b700 1 --2- 172.21.15.110:0/1412490983 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fee44067a80 0x7fee44069f30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fee60139dd0 tx=0x7fee50007ec0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:43.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.843+0000 7f7c362bd700 1 -- 172.21.15.110:0/908176074 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f7c300acd30 msgr2=0x7f7c3006e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.843+0000 7f7c362bd700 1 --2- 172.21.15.110:0/908176074 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f7c300acd30 0x7f7c3006e860 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7c28009f50 tx=0x7f7c28009840 comp rx=0 tx=0).stop 2024-09-15T03:36:43.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.843+0000 7f7c362bd700 1 -- 172.21.15.110:0/908176074 shutdown_connections 2024-09-15T03:36:43.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.843+0000 7f7c362bd700 1 --2- 172.21.15.110:0/908176074 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f7c300acd30 0x7f7c3006e860 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.843+0000 7f7c362bd700 1 -- 172.21.15.110:0/908176074 >> 172.21.15.110:0/908176074 conn(0x7f7c30067670 msgr2=0x7f7c30067a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:43.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.843+0000 7fa7498a3700 1 --2- 172.21.15.110:0/1230917431 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fa728067bd0 0x7fa72806a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:43.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.843+0000 7fa74a0a4700 1 --2- 172.21.15.110:0/1230917431 >> [v2:172.21.15.110:6816/22355,v1:172.21.15.110:6817/22355] conn(0x7fa7240015f0 0x7fa724003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:43.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.843+0000 7f7c362bd700 1 -- 172.21.15.110:0/908176074 shutdown_connections 2024-09-15T03:36:43.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.843+0000 7f7c362bd700 1 -- 172.21.15.110:0/908176074 wait complete. 2024-09-15T03:36:43.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.843+0000 7fa7498a3700 1 --2- 172.21.15.110:0/1230917431 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fa728067bd0 0x7fa72806a080 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa7400093a0 tx=0x7fa740017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:43.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.843+0000 7f7c362bd700 1 Processor -- start 2024-09-15T03:36:43.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.843+0000 7f7c362bd700 1 -- start start 2024-09-15T03:36:43.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.843+0000 7fa74a0a4700 1 --2- 172.21.15.110:0/1230917431 >> [v2:172.21.15.110:6816/22355,v1:172.21.15.110:6817/22355] conn(0x7fa7240015f0 0x7fa724003aa0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:43.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.843+0000 7f7c362bd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7c300acd30 0x7f7c3013bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:43.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.843+0000 7f7c362bd700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f7c300ad100 0x7f7c300af060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:43.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.843+0000 7f7c362bd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c30139ad0 0x7f7c3014ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:43.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.843+0000 7f7c362bd700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c300af5a0 con 0x7f7c30139ad0 2024-09-15T03:36:43.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.843+0000 7f7c362bd700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c300af850 con 0x7f7c300acd30 2024-09-15T03:36:43.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.843+0000 7f7c362bd700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3013e380 con 0x7f7c300ad100 2024-09-15T03:36:43.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.847+0000 7f7c35abc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c30139ad0 0x7f7c3014ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:43.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.847+0000 7f7c35abc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c30139ad0 0x7f7c3014ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37312/0 (socket says 172.21.15.110:37312) 2024-09-15T03:36:43.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.847+0000 7f7c35abc700 1 -- 172.21.15.110:0/3110709781 learned_addr learned my addr 172.21.15.110:0/3110709781 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:36:43.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.847+0000 7f7c34aba700 1 --2- 172.21.15.110:0/3110709781 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f7c300ad100 0x7f7c300af060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:43.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.847+0000 7f7c352bb700 1 --2- 172.21.15.110:0/3110709781 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7c300acd30 0x7f7c3013bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:43.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.847+0000 7fa739ffb700 1 -- 172.21.15.110:0/1230917431 <== osd.3 v2:172.21.15.110:6816/22355 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fa724006ae0 con 0x7fa7240015f0 2024-09-15T03:36:43.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.847+0000 7f7c35abc700 1 -- 172.21.15.110:0/3110709781 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f7c300ad100 msgr2=0x7f7c300af060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.847+0000 7f7c35abc700 1 --2- 172.21.15.110:0/3110709781 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f7c300ad100 0x7f7c300af060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.847+0000 7f7c35abc700 1 -- 172.21.15.110:0/3110709781 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7c300acd30 msgr2=0x7f7c3013bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.847+0000 7f7c35abc700 1 --2- 172.21.15.110:0/3110709781 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7c300acd30 0x7f7c3013bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.847+0000 7f7c35abc700 1 -- 172.21.15.110:0/3110709781 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c28009ae0 con 0x7f7c30139ad0 2024-09-15T03:36:43.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.847+0000 7f0356d72700 1 -- 172.21.15.110:0/4138983868 --> [v2:172.21.15.110:6808/22354,v1:172.21.15.110:6809/22354] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f035014ce50 con 0x7f033c06dfa0 2024-09-15T03:36:43.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.847+0000 7fb4d4a1e700 1 --2- 172.21.15.110:0/1996833262 >> [v2:172.21.15.177:6816/22428,v1:172.21.15.177:6817/22428] conn(0x7fb4d015cf10 0x7fb4d015d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:43.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.847+0000 7fb4d4a1e700 1 -- 172.21.15.110:0/1996833262 --> [v2:172.21.15.177:6816/22428,v1:172.21.15.177:6817/22428] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb4d00a9190 con 0x7fb4d015cf10 2024-09-15T03:36:43.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.847+0000 7f94fceea700 1 -- 172.21.15.110:0/453878828 --> [v2:172.21.15.177:6800/22423,v1:172.21.15.177:6801/22423] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f94f814c7a0 con 0x7f94d806de50 2024-09-15T03:36:43.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.851+0000 7f034cff9700 1 -- 172.21.15.110:0/4138983868 <== osd.1 v2:172.21.15.110:6808/22354 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f035014ce50 con 0x7f033c06dfa0 2024-09-15T03:36:43.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.851+0000 7fb4ced9d700 1 --2- 172.21.15.110:0/1996833262 >> [v2:172.21.15.177:6816/22428,v1:172.21.15.177:6817/22428] conn(0x7fb4d015cf10 0x7fb4d015d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:43.901 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.851+0000 7f94e67fc700 1 -- 172.21.15.110:0/453878828 <== osd.7 v2:172.21.15.177:6800/22423 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f94f814c7a0 con 0x7f94d806de50 2024-09-15T03:36:43.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.851+0000 7f94d7fff700 1 -- 172.21.15.110:0/453878828 >> [v2:172.21.15.177:6800/22423,v1:172.21.15.177:6801/22423] conn(0x7f94d806de50 msgr2=0x7f94d8070280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.851+0000 7f94d7fff700 1 --2- 172.21.15.110:0/453878828 >> [v2:172.21.15.177:6800/22423,v1:172.21.15.177:6801/22423] conn(0x7f94d806de50 0x7f94d8070280 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.851+0000 7f94d7fff700 1 -- 172.21.15.110:0/453878828 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f94d8067a80 msgr2=0x7f94d8069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.851+0000 7f94d7fff700 1 --2- 172.21.15.110:0/453878828 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f94d8067a80 0x7f94d8069f30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f94ec003a30 tx=0x7f94ec007ec0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.851+0000 7f94d7fff700 1 -- 172.21.15.110:0/453878828 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f94f813d120 msgr2=0x7f94f813abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.851+0000 7f94d7fff700 1 --2- 172.21.15.110:0/453878828 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f94f813d120 0x7f94f813abd0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f94e800c560 tx=0x7f94e8007eb0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.851+0000 7f94f6d9d700 1 -- 172.21.15.110:0/453878828 reap_dead start 2024-09-15T03:36:43.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.851+0000 7f94d7fff700 1 -- 172.21.15.110:0/453878828 shutdown_connections 2024-09-15T03:36:43.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.851+0000 7f94d7fff700 1 -- 172.21.15.110:0/453878828 >> 172.21.15.110:0/453878828 conn(0x7f94f8067670 msgr2=0x7f94f806e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:43.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.851+0000 7f94d7fff700 1 -- 172.21.15.110:0/453878828 shutdown_connections 2024-09-15T03:36:43.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.851+0000 7f94d7fff700 1 -- 172.21.15.110:0/453878828 wait complete. 2024-09-15T03:36:43.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.851+0000 7f0356d72700 1 -- 172.21.15.110:0/4138983868 >> [v2:172.21.15.110:6808/22354,v1:172.21.15.110:6809/22354] conn(0x7f033c06dfa0 msgr2=0x7f033c0703d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.851+0000 7f0356d72700 1 --2- 172.21.15.110:0/4138983868 >> [v2:172.21.15.110:6808/22354,v1:172.21.15.110:6809/22354] conn(0x7f033c06dfa0 0x7f033c0703d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.851+0000 7fb4ced9d700 1 --2- 172.21.15.110:0/1996833262 >> [v2:172.21.15.177:6816/22428,v1:172.21.15.177:6817/22428] conn(0x7fb4d015cf10 0x7fb4d015d2e0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:43.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.851+0000 7f0356d72700 1 -- 172.21.15.110:0/4138983868 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f033c067bd0 msgr2=0x7f033c06a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.851+0000 7f0356d72700 1 --2- 172.21.15.110:0/4138983868 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f033c067bd0 0x7f033c06a080 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f034000c010 tx=0x7f0340017000 comp rx=0 tx=0).stop 2024-09-15T03:36:43.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.851+0000 7f0356d72700 1 -- 172.21.15.110:0/4138983868 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f035013d120 msgr2=0x7f0350150140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.851+0000 7f0356d72700 1 --2- 172.21.15.110:0/4138983868 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f035013d120 0x7f0350150140 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f03500685a0 tx=0x7f0344008040 comp rx=0 tx=0).stop 2024-09-15T03:36:43.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.851+0000 7f035530f700 1 -- 172.21.15.110:0/4138983868 reap_dead start 2024-09-15T03:36:43.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.851+0000 7f0356d72700 1 -- 172.21.15.110:0/4138983868 shutdown_connections 2024-09-15T03:36:43.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.851+0000 7f0356d72700 1 -- 172.21.15.110:0/4138983868 >> 172.21.15.110:0/4138983868 conn(0x7f0350067670 msgr2=0x7f035006ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:43.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.851+0000 7f0356d72700 1 -- 172.21.15.110:0/4138983868 shutdown_connections 2024-09-15T03:36:43.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.851+0000 7f0356d72700 1 -- 172.21.15.110:0/4138983868 wait complete. 2024-09-15T03:36:43.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.847+0000 7f7c352bb700 1 --2- 172.21.15.110:0/3110709781 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7c300acd30 0x7f7c3013bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T03:36:43.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.847+0000 7f7c35abc700 1 --2- 172.21.15.110:0/3110709781 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c30139ad0 0x7f7c3014ced0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7c2000b4f0 tx=0x7f7c200090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:43.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.847+0000 7f7c257fa700 1 -- 172.21.15.110:0/3110709781 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c20009160 con 0x7f7c30139ad0 2024-09-15T03:36:43.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.847+0000 7f7c257fa700 1 -- 172.21.15.110:0/3110709781 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c2000f510 con 0x7f7c30139ad0 2024-09-15T03:36:43.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.847+0000 7f7c257fa700 1 -- 172.21.15.110:0/3110709781 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2000f8d0 con 0x7f7c30139ad0 2024-09-15T03:36:43.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.851+0000 7f7c362bd700 1 -- 172.21.15.110:0/3110709781 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c3013e5b0 con 0x7f7c30139ad0 2024-09-15T03:36:43.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.851+0000 7f7c362bd700 1 -- 172.21.15.110:0/3110709781 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c3014d680 con 0x7f7c30139ad0 2024-09-15T03:36:43.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.851+0000 7f7c362bd700 1 -- 172.21.15.110:0/3110709781 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7c300716d0 con 0x7f7c30139ad0 2024-09-15T03:36:43.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.851+0000 7f7c257fa700 1 -- 172.21.15.110:0/3110709781 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) v1 ==== 85847+0+0 (secure 0 0 0) 0x7f7c2000fc40 con 0x7f7c30139ad0 2024-09-15T03:36:43.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.851+0000 7f7c257fa700 1 --2- 172.21.15.110:0/3110709781 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f7c14067b50 0x7f7c1406a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:43.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.851+0000 7f7c352bb700 1 --2- 172.21.15.110:0/3110709781 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f7c14067b50 0x7f7c1406a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:43.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.851+0000 7f7c257fa700 1 -- 172.21.15.110:0/3110709781 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5262+0+0 (secure 0 0 0) 0x7f7c20084cb0 con 0x7f7c30139ad0 2024-09-15T03:36:43.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.851+0000 7f7c257fa700 1 --2- 172.21.15.110:0/3110709781 >> [v2:172.21.15.177:6802/22424,v1:172.21.15.177:6804/22424] conn(0x7f7c1407ef30 0x7f7c14081360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:43.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.851+0000 7f7c257fa700 1 -- 172.21.15.110:0/3110709781 --> [v2:172.21.15.177:6802/22424,v1:172.21.15.177:6804/22424] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7c14081a30 con 0x7f7c1407ef30 2024-09-15T03:36:43.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.851+0000 7f7c257fa700 1 -- 172.21.15.110:0/3110709781 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_get_version_reply(handle=1 version=19) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c200872e0 con 0x7f7c30139ad0 2024-09-15T03:36:43.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.851+0000 7f7c352bb700 1 --2- 172.21.15.110:0/3110709781 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f7c14067b50 0x7f7c1406a000 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7c28009ea0 tx=0x7f7c28017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:43.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.851+0000 7f7c34aba700 1 --2- 172.21.15.110:0/3110709781 >> [v2:172.21.15.177:6802/22424,v1:172.21.15.177:6804/22424] conn(0x7f7c1407ef30 0x7f7c14081360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:43.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.855+0000 7f7c34aba700 1 --2- 172.21.15.110:0/3110709781 >> [v2:172.21.15.177:6802/22424,v1:172.21.15.177:6804/22424] conn(0x7f7c1407ef30 0x7f7c14081360 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:43.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.855+0000 7f7c257fa700 1 -- 172.21.15.110:0/3110709781 <== osd.5 v2:172.21.15.177:6802/22424 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f7c14081a30 con 0x7f7c1407ef30 2024-09-15T03:36:43.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.855+0000 7fb4be7fc700 1 -- 172.21.15.110:0/1996833262 <== osd.6 v2:172.21.15.177:6816/22428 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fb4d00a9190 con 0x7fb4d015cf10 2024-09-15T03:36:43.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.863+0000 7fb492ca4700 1 -- 172.21.15.110:0/1374622977 --> [v2:172.21.15.110:6800/22352,v1:172.21.15.110:6801/22352] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb48c0b0710 con 0x7fb47806df00 2024-09-15T03:36:43.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.863+0000 7fb488ff9700 1 -- 172.21.15.110:0/1374622977 <== osd.0 v2:172.21.15.110:6800/22352 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb48c0b0710 con 0x7fb47806df00 2024-09-15T03:36:43.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.863+0000 7fb4727fc700 1 -- 172.21.15.110:0/1374622977 >> [v2:172.21.15.110:6800/22352,v1:172.21.15.110:6801/22352] conn(0x7fb47806df00 msgr2=0x7fb478070330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.863+0000 7fb4727fc700 1 --2- 172.21.15.110:0/1374622977 >> [v2:172.21.15.110:6800/22352,v1:172.21.15.110:6801/22352] conn(0x7fb47806df00 0x7fb478070330 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.863+0000 7fb4727fc700 1 -- 172.21.15.110:0/1374622977 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fb478067b30 msgr2=0x7fb478069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.863+0000 7fb4727fc700 1 --2- 172.21.15.110:0/1374622977 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fb478067b30 0x7fb478069fe0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb47c009500 tx=0x7fb47c0091a0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.863+0000 7fb4727fc700 1 -- 172.21.15.110:0/1374622977 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fb48c0acb70 msgr2=0x7fb48c079ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.863+0000 7fb4727fc700 1 --2- 172.21.15.110:0/1374622977 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fb48c0acb70 0x7fb48c079ec0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb480009bb0 tx=0x7fb480013000 comp rx=0 tx=0).stop 2024-09-15T03:36:43.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.863+0000 7fb491241700 1 -- 172.21.15.110:0/1374622977 reap_dead start 2024-09-15T03:36:43.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.863+0000 7fb4727fc700 1 -- 172.21.15.110:0/1374622977 shutdown_connections 2024-09-15T03:36:43.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.863+0000 7fb4727fc700 1 -- 172.21.15.110:0/1374622977 >> 172.21.15.110:0/1374622977 conn(0x7fb48c067670 msgr2=0x7fb48c0ab0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:43.910 INFO:teuthology.orchestra.run.smithi110.stdout:60129542148 2024-09-15T03:36:43.911 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-15T03:36:43.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.863+0000 7fb4727fc700 1 -- 172.21.15.110:0/1374622977 shutdown_connections 2024-09-15T03:36:43.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.863+0000 7fb4727fc700 1 -- 172.21.15.110:0/1374622977 wait complete. 2024-09-15T03:36:43.912 INFO:teuthology.orchestra.run.smithi110.stdout:55834574851 2024-09-15T03:36:43.912 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T03:36:43.912 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.867+0000 7fee6627e700 1 -- 172.21.15.110:0/1412490983 --> [v2:172.21.15.110:6822/22356,v1:172.21.15.110:6824/22356] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fee40002bd0 con 0x7fee4406de50 2024-09-15T03:36:43.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.867+0000 7fee9974a700 1 -- 172.21.15.110:0/1100349915 --> [v2:172.21.15.177:6822/22425,v1:172.21.15.177:6824/22425] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fee74002bd0 con 0x7fee8006dfa0 2024-09-15T03:36:43.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.867+0000 7fa74a8a5700 1 -- 172.21.15.110:0/1230917431 --> [v2:172.21.15.110:6816/22355,v1:172.21.15.110:6817/22355] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa724005bd0 con 0x7fa7240015f0 2024-09-15T03:36:43.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.867+0000 7fee557fa700 1 -- 172.21.15.110:0/1412490983 <== osd.2 v2:172.21.15.110:6822/22356 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fee40002bd0 con 0x7fee4406de50 2024-09-15T03:36:43.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.867+0000 7fee7affd700 1 -- 172.21.15.110:0/1100349915 <== osd.4 v2:172.21.15.177:6822/22425 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fee74002bd0 con 0x7fee8006dfa0 2024-09-15T03:36:43.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.867+0000 7fa739ffb700 1 -- 172.21.15.110:0/1230917431 <== osd.3 v2:172.21.15.110:6816/22355 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa724005bd0 con 0x7fa7240015f0 2024-09-15T03:36:43.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.871+0000 7fee6627e700 1 -- 172.21.15.110:0/1412490983 >> [v2:172.21.15.110:6822/22356,v1:172.21.15.110:6824/22356] conn(0x7fee4406de50 msgr2=0x7fee44070280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.871+0000 7fee6627e700 1 --2- 172.21.15.110:0/1412490983 >> [v2:172.21.15.110:6822/22356,v1:172.21.15.110:6824/22356] conn(0x7fee4406de50 0x7fee44070280 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.871+0000 7fee6627e700 1 -- 172.21.15.110:0/1412490983 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fee44067a80 msgr2=0x7fee44069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.871+0000 7fee6627e700 1 --2- 172.21.15.110:0/1412490983 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fee44067a80 0x7fee44069f30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fee60139dd0 tx=0x7fee50007ec0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.871+0000 7fee6627e700 1 -- 172.21.15.110:0/1412490983 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee600ab300 msgr2=0x7fee6014e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.871+0000 7fee6627e700 1 --2- 172.21.15.110:0/1412490983 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee600ab300 0x7fee6014e250 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fee5c00c7c0 tx=0x7fee5c007ef0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.871+0000 7fee65a7d700 1 -- 172.21.15.110:0/1412490983 reap_dead start 2024-09-15T03:36:43.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.871+0000 7fee6627e700 1 -- 172.21.15.110:0/1412490983 shutdown_connections 2024-09-15T03:36:43.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.871+0000 7fee6627e700 1 -- 172.21.15.110:0/1412490983 >> 172.21.15.110:0/1412490983 conn(0x7fee60067670 msgr2=0x7fee600b0780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:43.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.871+0000 7fee6627e700 1 -- 172.21.15.110:0/1412490983 shutdown_connections 2024-09-15T03:36:43.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.871+0000 7fee6627e700 1 -- 172.21.15.110:0/1412490983 wait complete. 2024-09-15T03:36:43.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.875+0000 7f7c362bd700 1 -- 172.21.15.110:0/3110709781 --> [v2:172.21.15.177:6802/22424,v1:172.21.15.177:6804/22424] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7c3014db40 con 0x7f7c1407ef30 2024-09-15T03:36:43.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.875+0000 7fee9974a700 1 -- 172.21.15.110:0/1100349915 >> [v2:172.21.15.177:6822/22425,v1:172.21.15.177:6824/22425] conn(0x7fee8006dfa0 msgr2=0x7fee800703d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.875+0000 7fee9974a700 1 --2- 172.21.15.110:0/1100349915 >> [v2:172.21.15.177:6822/22425,v1:172.21.15.177:6824/22425] conn(0x7fee8006dfa0 0x7fee800703d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.875+0000 7fa74a8a5700 1 -- 172.21.15.110:0/1230917431 >> [v2:172.21.15.110:6816/22355,v1:172.21.15.110:6817/22355] conn(0x7fa7240015f0 msgr2=0x7fa724003aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.875+0000 7fa74a8a5700 1 --2- 172.21.15.110:0/1230917431 >> [v2:172.21.15.110:6816/22355,v1:172.21.15.110:6817/22355] conn(0x7fa7240015f0 0x7fa724003aa0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.875+0000 7fee9974a700 1 -- 172.21.15.110:0/1100349915 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fee80067bd0 msgr2=0x7fee8006a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.875+0000 7fee9974a700 1 --2- 172.21.15.110:0/1100349915 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fee80067bd0 0x7fee8006a080 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fee8800aed0 tx=0x7fee88019000 comp rx=0 tx=0).stop 2024-09-15T03:36:43.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.875+0000 7fee9974a700 1 -- 172.21.15.110:0/1100349915 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fee940aef20 msgr2=0x7fee94154a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.875+0000 7fee9974a700 1 --2- 172.21.15.110:0/1100349915 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fee940aef20 0x7fee94154a30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fee9406ca40 tx=0x7fee8c0015f0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.875+0000 7fee93fff700 1 -- 172.21.15.110:0/1100349915 reap_dead start 2024-09-15T03:36:43.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.875+0000 7fa74a8a5700 1 -- 172.21.15.110:0/1230917431 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fa728067bd0 msgr2=0x7fa72806a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.875+0000 7fa74a8a5700 1 --2- 172.21.15.110:0/1230917431 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fa728067bd0 0x7fa72806a080 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa7400093a0 tx=0x7fa740017000 comp rx=0 tx=0).stop 2024-09-15T03:36:43.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.875+0000 7fa74a8a5700 1 -- 172.21.15.110:0/1230917431 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fa74406c260 msgr2=0x7fa7440a8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.875+0000 7fee9974a700 1 -- 172.21.15.110:0/1100349915 shutdown_connections 2024-09-15T03:36:43.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.875+0000 7fa74a8a5700 1 --2- 172.21.15.110:0/1230917431 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fa74406c260 0x7fa7440a8ab0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa73400c7c0 tx=0x7fa734007f70 comp rx=0 tx=0).stop 2024-09-15T03:36:43.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.875+0000 7fee9974a700 1 -- 172.21.15.110:0/1100349915 >> 172.21.15.110:0/1100349915 conn(0x7fee94069850 msgr2=0x7fee94138fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:43.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.875+0000 7fa74a0a4700 1 -- 172.21.15.110:0/1230917431 reap_dead start 2024-09-15T03:36:43.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.875+0000 7fa74a8a5700 1 -- 172.21.15.110:0/1230917431 shutdown_connections 2024-09-15T03:36:43.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.875+0000 7fa74a8a5700 1 -- 172.21.15.110:0/1230917431 >> 172.21.15.110:0/1230917431 conn(0x7fa744067810 msgr2=0x7fa74413cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:43.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.875+0000 7f7c257fa700 1 -- 172.21.15.110:0/3110709781 <== osd.5 v2:172.21.15.177:6802/22424 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7c3014db40 con 0x7f7c1407ef30 2024-09-15T03:36:43.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.875+0000 7fa74a8a5700 1 -- 172.21.15.110:0/1230917431 shutdown_connections 2024-09-15T03:36:43.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.875+0000 7fa74a8a5700 1 -- 172.21.15.110:0/1230917431 wait complete. 2024-09-15T03:36:43.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.875+0000 7fee9974a700 1 -- 172.21.15.110:0/1100349915 shutdown_connections 2024-09-15T03:36:43.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.875+0000 7fee9974a700 1 -- 172.21.15.110:0/1100349915 wait complete. 2024-09-15T03:36:43.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.875+0000 7f7c362bd700 1 -- 172.21.15.110:0/3110709781 >> [v2:172.21.15.177:6802/22424,v1:172.21.15.177:6804/22424] conn(0x7f7c1407ef30 msgr2=0x7f7c14081360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.875+0000 7f7c362bd700 1 --2- 172.21.15.110:0/3110709781 >> [v2:172.21.15.177:6802/22424,v1:172.21.15.177:6804/22424] conn(0x7f7c1407ef30 0x7f7c14081360 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.875+0000 7f7c362bd700 1 -- 172.21.15.110:0/3110709781 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f7c14067b50 msgr2=0x7f7c1406a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.875+0000 7f7c362bd700 1 --2- 172.21.15.110:0/3110709781 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f7c14067b50 0x7f7c1406a000 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7c28009ea0 tx=0x7f7c28017000 comp rx=0 tx=0).stop 2024-09-15T03:36:43.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.875+0000 7f7c362bd700 1 -- 172.21.15.110:0/3110709781 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c30139ad0 msgr2=0x7f7c3014ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.875+0000 7f7c362bd700 1 --2- 172.21.15.110:0/3110709781 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c30139ad0 0x7f7c3014ced0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7c2000b4f0 tx=0x7f7c200090f0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.875+0000 7f7c35abc700 1 -- 172.21.15.110:0/3110709781 reap_dead start 2024-09-15T03:36:43.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.875+0000 7f7c362bd700 1 -- 172.21.15.110:0/3110709781 shutdown_connections 2024-09-15T03:36:43.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.875+0000 7f7c362bd700 1 -- 172.21.15.110:0/3110709781 >> 172.21.15.110:0/3110709781 conn(0x7f7c30067670 msgr2=0x7f7c300a9330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:43.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.875+0000 7f7c362bd700 1 -- 172.21.15.110:0/3110709781 shutdown_connections 2024-09-15T03:36:43.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.875+0000 7f7c362bd700 1 -- 172.21.15.110:0/3110709781 wait complete. 2024-09-15T03:36:43.923 INFO:teuthology.orchestra.run.smithi110.stdout:55834574852 2024-09-15T03:36:43.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.879+0000 7fb4d4a1e700 1 -- 172.21.15.110:0/1996833262 --> [v2:172.21.15.177:6816/22428,v1:172.21.15.177:6817/22428] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb4d0064bd0 con 0x7fb4d015cf10 2024-09-15T03:36:43.924 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T03:36:43.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.879+0000 7fb4be7fc700 1 -- 172.21.15.110:0/1996833262 <== osd.6 v2:172.21.15.177:6816/22428 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb4d0064bd0 con 0x7fb4d015cf10 2024-09-15T03:36:43.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.879+0000 7fb4d4a1e700 1 -- 172.21.15.110:0/1996833262 >> [v2:172.21.15.177:6816/22428,v1:172.21.15.177:6817/22428] conn(0x7fb4d015cf10 msgr2=0x7fb4d015d2e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.879+0000 7fb4d4a1e700 1 --2- 172.21.15.110:0/1996833262 >> [v2:172.21.15.177:6816/22428,v1:172.21.15.177:6817/22428] conn(0x7fb4d015cf10 0x7fb4d015d2e0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:43.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.879+0000 7fb4d4a1e700 1 -- 172.21.15.110:0/1996833262 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fb4b0067a80 msgr2=0x7fb4b0069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.879+0000 7fb4d4a1e700 1 --2- 172.21.15.110:0/1996833262 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fb4b0067a80 0x7fb4b0069f30 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb4c80145d0 tx=0x7fb4c801d000 comp rx=0 tx=0).stop 2024-09-15T03:36:43.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.879+0000 7fb4d4a1e700 1 -- 172.21.15.110:0/1996833262 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4d0086310 msgr2=0x7fb4d00a86b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:43.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.879+0000 7fb4d4a1e700 1 --2- 172.21.15.110:0/1996833262 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4d0086310 0x7fb4d00a86b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb4c000d960 tx=0x7fb4c0009450 comp rx=0 tx=0).stop 2024-09-15T03:36:43.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.879+0000 7fb4ced9d700 1 -- 172.21.15.110:0/1996833262 reap_dead start 2024-09-15T03:36:43.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.879+0000 7fb4d4a1e700 1 -- 172.21.15.110:0/1996833262 shutdown_connections 2024-09-15T03:36:43.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.879+0000 7fb4d4a1e700 1 -- 172.21.15.110:0/1996833262 >> 172.21.15.110:0/1996833262 conn(0x7fb4d0067670 msgr2=0x7fb4d00ad920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:43.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.879+0000 7fb4d4a1e700 1 -- 172.21.15.110:0/1996833262 shutdown_connections 2024-09-15T03:36:43.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:43.879+0000 7fb4d4a1e700 1 -- 172.21.15.110:0/1996833262 wait complete. 2024-09-15T03:36:43.927 INFO:teuthology.orchestra.run.smithi110.stdout:60129542148 2024-09-15T03:36:43.927 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T03:36:43.928 INFO:teuthology.orchestra.run.smithi110.stdout:60129542147 2024-09-15T03:36:43.928 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-15T03:36:43.929 INFO:teuthology.orchestra.run.smithi110.stdout:60129542147 2024-09-15T03:36:43.929 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-15T03:36:43.930 INFO:teuthology.orchestra.run.smithi110.stdout:55834574852 2024-09-15T03:36:43.930 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-15T03:36:43.931 INFO:teuthology.orchestra.run.smithi110.stdout:60129542147 2024-09-15T03:36:43.932 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-15T03:36:44.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.107+0000 7f727da78700 1 Processor -- start 2024-09-15T03:36:44.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.107+0000 7f727da78700 1 -- start start 2024-09-15T03:36:44.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.107+0000 7f727da78700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f72780acc40 0x7f7278141370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:44.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.107+0000 7f727da78700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f72780ad010 0x7f727806e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:44.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.107+0000 7f727da78700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f727806eba0 0x7f727813d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:44.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.107+0000 7f727da78700 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72780ac540 con 0x7f72780ad010 2024-09-15T03:36:44.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.107+0000 7f727da78700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f727806f000 con 0x7f727806eba0 2024-09-15T03:36:44.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.107+0000 7f727da78700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f727806bb10 con 0x7f72780acc40 2024-09-15T03:36:44.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.107+0000 7f727ca76700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f72780acc40 0x7f7278141370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:44.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.107+0000 7f727ca76700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f72780acc40 0x7f7278141370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3301/0 says I am v2:172.21.15.110:44802/0 (socket says 172.21.15.110:44802) 2024-09-15T03:36:44.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.107+0000 7f727ca76700 1 -- 172.21.15.110:0/2176873889 learned_addr learned my addr 172.21.15.110:0/2176873889 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:36:44.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.107+0000 7f7277fff700 1 --2- 172.21.15.110:0/2176873889 >> v2:172.21.15.110:3300/0 conn(0x7f72780ad010 0x7f727806e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:44.113 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.107+0000 7f727ca76700 1 -- 172.21.15.110:0/2176873889 >> v2:172.21.15.177:3300/0 conn(0x7f727806eba0 msgr2=0x7f727813d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:36:44.113 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.107+0000 7f727ca76700 1 --2- 172.21.15.110:0/2176873889 >> v2:172.21.15.177:3300/0 conn(0x7f727806eba0 0x7f727813d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.113 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.107+0000 7f727ca76700 1 -- 172.21.15.110:0/2176873889 >> v2:172.21.15.110:3300/0 conn(0x7f72780ad010 msgr2=0x7f727806e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:36:44.113 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.107+0000 7f727ca76700 1 --2- 172.21.15.110:0/2176873889 >> v2:172.21.15.110:3300/0 conn(0x7f72780ad010 0x7f727806e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.113 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.107+0000 7f727ca76700 1 -- 172.21.15.110:0/2176873889 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f727806bdf0 con 0x7f72780acc40 2024-09-15T03:36:44.113 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.107+0000 7f727ca76700 1 --2- 172.21.15.110:0/2176873889 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f72780acc40 0x7f7278141370 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7278137ee0 tx=0x7f7268009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acc7ad5a1f6fe86f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:44.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.107+0000 7f646b6b2700 1 Processor -- start 2024-09-15T03:36:44.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.107+0000 7f646b6b2700 1 -- start start 2024-09-15T03:36:44.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.107+0000 7f72777fe700 1 -- 172.21.15.110:0/2176873889 <== mon.2 v2:172.21.15.110:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7268009e70 con 0x7f72780acc40 2024-09-15T03:36:44.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.107+0000 7f72777fe700 1 -- 172.21.15.110:0/2176873889 <== mon.2 v2:172.21.15.110:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f726800b860 con 0x7f72780acc40 2024-09-15T03:36:44.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.107+0000 7f72777fe700 1 -- 172.21.15.110:0/2176873889 <== mon.2 v2:172.21.15.110:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726800bc20 con 0x7f72780acc40 2024-09-15T03:36:44.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.107+0000 7f646b6b2700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f64640ad4f0 0x7f6464139e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:44.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.107+0000 7f646b6b2700 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6464139740 con 0x7f64640ad120 2024-09-15T03:36:44.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.107+0000 7f646b6b2700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f646406f200 con 0x7f646413a390 2024-09-15T03:36:44.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.107+0000 7f646b6b2700 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f646406da20 con 0x7f64640ad4f0 2024-09-15T03:36:44.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.111+0000 7f646944e700 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f64640ad120 0x7f646406ea10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:39958/0 (socket says 172.21.15.110:39958) 2024-09-15T03:36:44.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.111+0000 7f646944e700 1 -- 172.21.15.110:0/1545088041 learned_addr learned my addr 172.21.15.110:0/1545088041 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-15T03:36:44.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.111+0000 7f6468c4d700 1 --2- 172.21.15.110:0/1545088041 >> v2:172.21.15.110:3300/0 conn(0x7f64640ad4f0 0x7f6464139e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:44.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.111+0000 7f727da78700 1 -- 172.21.15.110:0/2176873889 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f72780acc40 msgr2=0x7f7278141370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:44.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.111+0000 7f727da78700 1 --2- 172.21.15.110:0/2176873889 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f72780acc40 0x7f7278141370 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7278137ee0 tx=0x7f7268009250 comp rx=0 tx=0).stop 2024-09-15T03:36:44.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.111+0000 7feafa207700 1 Processor -- start 2024-09-15T03:36:44.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.111+0000 7feafa207700 1 -- start start 2024-09-15T03:36:44.117 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.111+0000 7f6468c4d700 1 -- 172.21.15.110:0/1545088041 >> v1:172.21.15.177:6789/0 conn(0x7f646413a390 legacy=0x7f646413d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:44.117 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.111+0000 7f6463fff700 1 -- 172.21.15.110:0/1545088041 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3822959772 0 0) 0x7f6464139740 con 0x7f64640ad120 2024-09-15T03:36:44.117 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.111+0000 7f6468c4d700 1 -- 172.21.15.110:0/1545088041 >> v1:172.21.15.110:6789/0 conn(0x7f64640ad120 legacy=0x7f646406ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:44.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.111+0000 7f6468c4d700 1 -- 172.21.15.110:0/1545088041 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6464137310 con 0x7f64640ad4f0 2024-09-15T03:36:44.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.111+0000 7f6468c4d700 1 --2- 172.21.15.110:0/1545088041 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f64640ad4f0 0x7f6464139e50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f645800c640 tx=0x7f6458007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f13b302304c19cb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:44.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.111+0000 7f6463fff700 1 -- 172.21.15.110:0/1545088041 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645800a6f0 con 0x7f64640ad4f0 2024-09-15T03:36:44.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.111+0000 7f6463fff700 1 -- 172.21.15.110:0/1545088041 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f645800a850 con 0x7f64640ad4f0 2024-09-15T03:36:44.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.111+0000 7f6463fff700 1 -- 172.21.15.110:0/1545088041 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645800abf0 con 0x7f64640ad4f0 2024-09-15T03:36:44.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.111+0000 7feafa207700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7feaf40ad390 0x7feaf406cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:44.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.111+0000 7feafa207700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7feaf41371d0 0x7feaf4137600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:44.118 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.111+0000 7feafa207700 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaf406d750 con 0x7feaf413da10 2024-09-15T03:36:44.119 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.111+0000 7f727da78700 1 -- 172.21.15.110:0/2176873889 shutdown_connections 2024-09-15T03:36:44.119 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.111+0000 7f727da78700 1 --2- 172.21.15.110:0/2176873889 >> v2:172.21.15.177:3300/0 conn(0x7f727806eba0 0x7f727813d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.119 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.111+0000 7f727da78700 1 --2- 172.21.15.110:0/2176873889 >> v2:172.21.15.110:3300/0 conn(0x7f72780ad010 0x7f727806e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.119 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.111+0000 7f727da78700 1 --2- 172.21.15.110:0/2176873889 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f72780acc40 0x7f7278141370 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.119 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.111+0000 7f727da78700 1 -- 172.21.15.110:0/2176873889 >> 172.21.15.110:0/2176873889 conn(0x7f7278067670 msgr2=0x7f7278067a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:44.120 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.111+0000 7feafa207700 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaf413e530 con 0x7feaf41371d0 2024-09-15T03:36:44.120 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.111+0000 7feafa207700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaf406d450 con 0x7feaf40ad390 2024-09-15T03:36:44.120 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.111+0000 7feaf3fff700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7feaf41371d0 0x7feaf4137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:44.120 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.111+0000 7feaf2ffd700 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7feaf413da10 0x7feaf413de20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:39964/0 (socket says 172.21.15.110:39964) 2024-09-15T03:36:44.120 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.111+0000 7feaf2ffd700 1 -- 172.21.15.110:0/1398223825 learned_addr learned my addr 172.21.15.110:0/1398223825 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-15T03:36:44.120 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.111+0000 7f727da78700 1 -- 172.21.15.110:0/2176873889 shutdown_connections 2024-09-15T03:36:44.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.111+0000 7f727da78700 1 -- 172.21.15.110:0/2176873889 wait complete. 2024-09-15T03:36:44.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.111+0000 7feaf37fe700 1 --2- 172.21.15.110:0/1398223825 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7feaf40ad390 0x7feaf406cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:44.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.111+0000 7feaf27fc700 1 -- 172.21.15.110:0/1398223825 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2122823737 0 0) 0x7feaf406d750 con 0x7feaf413da10 2024-09-15T03:36:44.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.111+0000 7feaf27fc700 1 -- 172.21.15.110:0/1398223825 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feadc003570 con 0x7feaf413da10 2024-09-15T03:36:44.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.111+0000 7f1d10072700 1 Processor -- start 2024-09-15T03:36:44.122 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.111+0000 7f646b6b2700 1 -- 172.21.15.110:0/1545088041 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f64640ad4f0 msgr2=0x7f6464139e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:44.122 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.111+0000 7f646b6b2700 1 --2- 172.21.15.110:0/1545088041 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f64640ad4f0 0x7f6464139e50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f645800c640 tx=0x7f6458007a40 comp rx=0 tx=0).stop 2024-09-15T03:36:44.122 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.111+0000 7f646b6b2700 1 -- 172.21.15.110:0/1545088041 shutdown_connections 2024-09-15T03:36:44.122 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.111+0000 7f646b6b2700 1 --2- 172.21.15.110:0/1545088041 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f64640ad4f0 0x7f6464139e50 secure :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f645800c640 tx=0x7f6458007a40 comp rx=0 tx=0).stop 2024-09-15T03:36:44.122 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.111+0000 7f646b6b2700 1 -- 172.21.15.110:0/1545088041 >> 172.21.15.110:0/1545088041 conn(0x7f6464067a10 msgr2=0x7f6464067e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:44.122 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.111+0000 7f1d10072700 1 -- start start 2024-09-15T03:36:44.123 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.111+0000 7feaf37fe700 1 -- 172.21.15.110:0/1398223825 >> v2:172.21.15.110:3300/0 conn(0x7feaf41371d0 msgr2=0x7feaf4137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:36:44.123 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.111+0000 7feaf37fe700 1 --2- 172.21.15.110:0/1398223825 >> v2:172.21.15.110:3300/0 conn(0x7feaf41371d0 0x7feaf4137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.123 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.111+0000 7feaf37fe700 1 -- 172.21.15.110:0/1398223825 >> v1:172.21.15.110:6789/0 conn(0x7feaf413da10 legacy=0x7feaf413de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:44.123 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.111+0000 7f646b6b2700 1 -- 172.21.15.110:0/1545088041 shutdown_connections 2024-09-15T03:36:44.123 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.111+0000 7feaf37fe700 1 -- 172.21.15.110:0/1398223825 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf40ad760 con 0x7feaf40ad390 2024-09-15T03:36:44.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.111+0000 7f646b6b2700 1 -- 172.21.15.110:0/1545088041 wait complete. 2024-09-15T03:36:44.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.111+0000 7feaf37fe700 1 --2- 172.21.15.110:0/1398223825 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7feaf40ad390 0x7feaf406cf10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7feae80098e0 tx=0x7feae80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=323abe3c9454a9cf server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:44.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.111+0000 7feaf27fc700 1 -- 172.21.15.110:0/1398223825 <== mon.2 v2:172.21.15.110:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae8011630 con 0x7feaf40ad390 2024-09-15T03:36:44.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.111+0000 7feaf27fc700 1 -- 172.21.15.110:0/1398223825 <== mon.2 v2:172.21.15.110:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feae80117b0 con 0x7feaf40ad390 2024-09-15T03:36:44.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.111+0000 7feaf27fc700 1 -- 172.21.15.110:0/1398223825 <== mon.2 v2:172.21.15.110:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae8011b70 con 0x7feaf40ad390 2024-09-15T03:36:44.125 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.111+0000 7f727da78700 1 Processor -- start 2024-09-15T03:36:44.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.111+0000 7f1d10072700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f1d080ad0f0 0x7f1d080a93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:44.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.111+0000 7f1d10072700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f1d080ad4c0 0x7f1d0813eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:44.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.111+0000 7f1d10072700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d080a4a00 con 0x7f1d080a9900 2024-09-15T03:36:44.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.111+0000 7f1d10072700 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d080a4800 con 0x7f1d080ad0f0 2024-09-15T03:36:44.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.111+0000 7f1d10072700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0813f780 con 0x7f1d080ad4c0 2024-09-15T03:36:44.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.111+0000 7f1d0de0e700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f1d080ad0f0 0x7f1d080a93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:44.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.111+0000 7f1d0d60d700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f1d080ad4c0 0x7f1d0813eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:44.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.111+0000 7f1d0d60d700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f1d080ad4c0 0x7f1d0813eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3301/0 says I am v2:172.21.15.110:44826/0 (socket says 172.21.15.110:44826) 2024-09-15T03:36:44.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.111+0000 7f1d0de0e700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f1d080ad0f0 0x7f1d080a93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37346/0 (socket says 172.21.15.110:37346) 2024-09-15T03:36:44.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.111+0000 7f1d0d60d700 1 -- 172.21.15.110:0/1836684240 learned_addr learned my addr 172.21.15.110:0/1836684240 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:36:44.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.111+0000 7f1d0d60d700 1 -- 172.21.15.110:0/1836684240 >> v2:172.21.15.110:3300/0 conn(0x7f1d080ad0f0 msgr2=0x7f1d080a93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:36:44.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f646b6b2700 1 Processor -- start 2024-09-15T03:36:44.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f1d0d60d700 1 --2- 172.21.15.110:0/1836684240 >> v2:172.21.15.110:3300/0 conn(0x7f1d080ad0f0 0x7f1d080a93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f1d0d60d700 1 -- 172.21.15.110:0/1836684240 >> v1:172.21.15.177:6789/0 conn(0x7f1d080a9900 legacy=0x7f1d0813f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:44.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f1d0d60d700 1 -- 172.21.15.110:0/1836684240 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d0813c0a0 con 0x7f1d080ad4c0 2024-09-15T03:36:44.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f646b6b2700 1 -- start start 2024-09-15T03:36:44.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7feafa207700 1 -- 172.21.15.110:0/1398223825 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7feaf40ad390 msgr2=0x7feaf406cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:44.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7feafa207700 1 --2- 172.21.15.110:0/1398223825 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7feaf40ad390 0x7feaf406cf10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7feae80098e0 tx=0x7feae80091a0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f727da78700 1 -- start start 2024-09-15T03:36:44.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f1d0de0e700 1 --2- 172.21.15.110:0/1836684240 >> v2:172.21.15.110:3300/0 conn(0x7f1d080ad0f0 0x7f1d080a93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T03:36:44.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f1d0d60d700 1 --2- 172.21.15.110:0/1836684240 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f1d080ad4c0 0x7f1d0813eb30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1cf80099b0 tx=0x7f1cf8009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d66949575afa6637 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:44.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f1d0ce0c700 1 -- 172.21.15.110:0/1836684240 <== mon.2 v2:172.21.15.110:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf8009ea0 con 0x7f1d080ad4c0 2024-09-15T03:36:44.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f1d0ce0c700 1 -- 172.21.15.110:0/1836684240 <== mon.2 v2:172.21.15.110:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cf800b860 con 0x7f1d080ad4c0 2024-09-15T03:36:44.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f1d0ce0c700 1 -- 172.21.15.110:0/1836684240 <== mon.2 v2:172.21.15.110:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf800bc20 con 0x7f1d080ad4c0 2024-09-15T03:36:44.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f727da78700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f727806eba0 0x7f7278144810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:44.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f646b6b2700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f64640ad120 0x7f64640af0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:44.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f727da78700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f72780acc40 0x7f727813a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:44.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f727da78700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72780ad010 0x7f7278138820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:44.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f3fe8f3b700 1 Processor -- start 2024-09-15T03:36:44.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f727da78700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f727813ae10 con 0x7f72780ad010 2024-09-15T03:36:44.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f727da78700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72780b0870 con 0x7f72780acc40 2024-09-15T03:36:44.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f727da78700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72780b0a30 con 0x7f727806eba0 2024-09-15T03:36:44.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f3fe8f3b700 1 -- start start 2024-09-15T03:36:44.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f646b6b2700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f646413a390 0x7f646413c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:44.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f727d277700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72780ad010 0x7f7278138820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:44.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f7277fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f72780acc40 0x7f727813a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:44.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f727d277700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72780ad010 0x7f7278138820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37358/0 (socket says 172.21.15.110:37358) 2024-09-15T03:36:44.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f727d277700 1 -- 172.21.15.110:0/4217961655 learned_addr learned my addr 172.21.15.110:0/4217961655 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:36:44.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f646b6b2700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64640a4d60 0x7f646414d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:44.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f646b6b2700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f646413caf0 con 0x7f646413a390 2024-09-15T03:36:44.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f646b6b2700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64640a5d30 con 0x7f64640a4d60 2024-09-15T03:36:44.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f646b6b2700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64640a4840 con 0x7f64640ad120 2024-09-15T03:36:44.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7feafa207700 1 -- 172.21.15.110:0/1398223825 shutdown_connections 2024-09-15T03:36:44.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7feafa207700 1 --2- 172.21.15.110:0/1398223825 >> v2:172.21.15.110:3300/0 conn(0x7feaf41371d0 0x7feaf4137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7feafa207700 1 --2- 172.21.15.110:0/1398223825 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7feaf40ad390 0x7feaf406cf10 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7feafa207700 1 -- 172.21.15.110:0/1398223825 >> 172.21.15.110:0/1398223825 conn(0x7feaf4067670 msgr2=0x7feaf4067a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:44.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7feafa207700 1 -- 172.21.15.110:0/1398223825 shutdown_connections 2024-09-15T03:36:44.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7feafa207700 1 -- 172.21.15.110:0/1398223825 wait complete. 2024-09-15T03:36:44.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f7277fff700 1 -- 172.21.15.110:0/4217961655 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f727806eba0 msgr2=0x7f7278144810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:44.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f7277fff700 1 --2- 172.21.15.110:0/4217961655 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f727806eba0 0x7f7278144810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f7277fff700 1 -- 172.21.15.110:0/4217961655 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72780ad010 msgr2=0x7f7278138820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:44.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f7277fff700 1 --2- 172.21.15.110:0/4217961655 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72780ad010 0x7f7278138820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f7277fff700 1 -- 172.21.15.110:0/4217961655 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72680094f0 con 0x7f72780acc40 2024-09-15T03:36:44.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f3fe8f3b700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f3fe40aca20 0x7f3fe40ad390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:44.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f646944e700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f64640ad120 0x7f64640af0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:44.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f646944e700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f64640ad120 0x7f64640af0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3301/0 says I am v2:172.21.15.110:44828/0 (socket says 172.21.15.110:44828) 2024-09-15T03:36:44.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f646944e700 1 -- 172.21.15.110:0/1563022706 learned_addr learned my addr 172.21.15.110:0/1563022706 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:36:44.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f6468c4d700 1 --2- 172.21.15.110:0/1563022706 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f646413a390 0x7f646413c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:44.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f3fe8f3b700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f3fe406bda0 0x7f3fe406e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:44.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f3fe8f3b700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f3fe40aa920 0x7f3fe40aada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:44.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f3fe8f3b700 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fe406cca0 con 0x7f3fe40aa920 2024-09-15T03:36:44.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f3fe8f3b700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fe406ca20 con 0x7f3fe406bda0 2024-09-15T03:36:44.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f3fe8f3b700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe406dd50 con 0x7f3fe40aca20 2024-09-15T03:36:44.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f6469c4f700 1 --2- 172.21.15.110:0/1563022706 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64640a4d60 0x7f646414d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:44.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f7277fff700 1 --2- 172.21.15.110:0/4217961655 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f72780acc40 0x7f727813a6d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f726c00ea50 tx=0x7f726c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:44.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f3fe259c700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f3fe40aca20 0x7f3fe40ad390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:44.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f3fe259c700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f3fe40aca20 0x7f3fe40ad390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3301/0 says I am v2:172.21.15.110:44852/0 (socket says 172.21.15.110:44852) 2024-09-15T03:36:44.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f646944e700 1 -- 172.21.15.110:0/1563022706 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64640a4d60 msgr2=0x7f646414d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:44.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f646944e700 1 --2- 172.21.15.110:0/1563022706 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64640a4d60 0x7f646414d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f646944e700 1 -- 172.21.15.110:0/1563022706 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f646413a390 msgr2=0x7f646413c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:44.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f646944e700 1 --2- 172.21.15.110:0/1563022706 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f646413a390 0x7f646413c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f646944e700 1 -- 172.21.15.110:0/1563022706 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6458007ce0 con 0x7f64640ad120 2024-09-15T03:36:44.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f6468c4d700 1 --2- 172.21.15.110:0/1563022706 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f646413a390 0x7f646413c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T03:36:44.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f3fe1d9b700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f3fe406bda0 0x7f3fe406e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:44.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f3fe1d9b700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f3fe406bda0 0x7f3fe406e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.110:49546/0 (socket says 172.21.15.110:49546) 2024-09-15T03:36:44.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f3fe1d9b700 1 -- 172.21.15.110:0/3077423658 learned_addr learned my addr 172.21.15.110:0/3077423658 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:36:44.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f3fe2d9d700 1 --2- 172.21.15.110:0/3077423658 >> v2:172.21.15.110:3300/0 conn(0x7f3fe40aa920 0x7f3fe40aada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:44.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f1d10072700 1 -- 172.21.15.110:0/1836684240 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f1d080ad4c0 msgr2=0x7f1d0813eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:44.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f1d10072700 1 --2- 172.21.15.110:0/1836684240 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f1d080ad4c0 0x7f1d0813eb30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1cf80099b0 tx=0x7f1cf8009250 comp rx=0 tx=0).stop 2024-09-15T03:36:44.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f1d10072700 1 -- 172.21.15.110:0/1836684240 shutdown_connections 2024-09-15T03:36:44.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f1d10072700 1 --2- 172.21.15.110:0/1836684240 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f1d080ad4c0 0x7f1d0813eb30 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f1d10072700 1 --2- 172.21.15.110:0/1836684240 >> v2:172.21.15.110:3300/0 conn(0x7f1d080ad0f0 0x7f1d080a93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f1d10072700 1 -- 172.21.15.110:0/1836684240 >> 172.21.15.110:0/1836684240 conn(0x7f1d08067670 msgr2=0x7f1d08067a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:44.140 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f3fe259c700 1 -- 172.21.15.110:0/3077423658 >> v2:172.21.15.177:3300/0 conn(0x7f3fe406bda0 msgr2=0x7f3fe406e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:36:44.140 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f3fe259c700 1 --2- 172.21.15.110:0/3077423658 >> v2:172.21.15.177:3300/0 conn(0x7f3fe406bda0 0x7f3fe406e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.140 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f3fe259c700 1 -- 172.21.15.110:0/3077423658 >> v2:172.21.15.110:3300/0 conn(0x7f3fe40aa920 msgr2=0x7f3fe40aada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:36:44.140 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f3fe259c700 1 --2- 172.21.15.110:0/3077423658 >> v2:172.21.15.110:3300/0 conn(0x7f3fe40aa920 0x7f3fe40aada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.140 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f3fe259c700 1 -- 172.21.15.110:0/3077423658 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe406e030 con 0x7f3fe40aca20 2024-09-15T03:36:44.140 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7feafa207700 1 Processor -- start 2024-09-15T03:36:44.140 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7feafa207700 1 -- start start 2024-09-15T03:36:44.141 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f646944e700 1 --2- 172.21.15.110:0/1563022706 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f64640ad120 0x7f64640af0e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f645c00c2a0 tx=0x7f645c001b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:44.141 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f64617fa700 1 -- 172.21.15.110:0/1563022706 <== mon.2 v2:172.21.15.110:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645c00cde0 con 0x7f64640ad120 2024-09-15T03:36:44.141 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f64617fa700 1 -- 172.21.15.110:0/1563022706 <== mon.2 v2:172.21.15.110:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f645c011470 con 0x7f64640ad120 2024-09-15T03:36:44.141 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f64617fa700 1 -- 172.21.15.110:0/1563022706 <== mon.2 v2:172.21.15.110:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645c0117f0 con 0x7f64640ad120 2024-09-15T03:36:44.141 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f3fe2d9d700 1 --2- 172.21.15.110:0/3077423658 >> v2:172.21.15.110:3300/0 conn(0x7f3fe40aa920 0x7f3fe40aada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T03:36:44.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.115+0000 7f3fe259c700 1 --2- 172.21.15.110:0/3077423658 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f3fe40aca20 0x7f3fe40ad390 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3fe40afe40 tx=0x7f3fd40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42daf5e95f229b86 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:44.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.119+0000 7feafa207700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feaf40ad390 0x7feaf4132d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:44.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.119+0000 7feafa207700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feaf41371d0 0x7feaf4133650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:44.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.119+0000 7feafa207700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7feaf413da10 0x7feaf415d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:44.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.119+0000 7feafa207700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf4134240 con 0x7feaf41371d0 2024-09-15T03:36:44.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.119+0000 7feaf2ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feaf41371d0 0x7feaf4133650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:44.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.119+0000 7feafa207700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf4133db0 con 0x7feaf40ad390 2024-09-15T03:36:44.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.119+0000 7feafa207700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaf40a4c30 con 0x7feaf413da10 2024-09-15T03:36:44.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.119+0000 7feaf2ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feaf41371d0 0x7feaf4133650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37388/0 (socket says 172.21.15.110:37388) 2024-09-15T03:36:44.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.119+0000 7f646b6b2700 1 -- 172.21.15.110:0/1563022706 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64640a44d0 con 0x7f64640ad120 2024-09-15T03:36:44.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.119+0000 7feaf2ffd700 1 -- 172.21.15.110:0/1027008740 learned_addr learned my addr 172.21.15.110:0/1027008740 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:36:44.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.119+0000 7f646b6b2700 1 -- 172.21.15.110:0/1563022706 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f646414d980 con 0x7f64640ad120 2024-09-15T03:36:44.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.119+0000 7f1d10072700 1 -- 172.21.15.110:0/1836684240 shutdown_connections 2024-09-15T03:36:44.145 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.119+0000 7feaf3fff700 1 --2- 172.21.15.110:0/1027008740 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7feaf413da10 0x7feaf415d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:44.145 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.119+0000 7feaf2ffd700 1 -- 172.21.15.110:0/1027008740 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7feaf413da10 msgr2=0x7feaf415d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:44.145 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.119+0000 7feaf2ffd700 1 --2- 172.21.15.110:0/1027008740 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7feaf413da10 0x7feaf415d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.145 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.119+0000 7feaf2ffd700 1 -- 172.21.15.110:0/1027008740 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feaf40ad390 msgr2=0x7feaf4132d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:44.145 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.119+0000 7feaf2ffd700 1 --2- 172.21.15.110:0/1027008740 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feaf40ad390 0x7feaf4132d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.145 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.119+0000 7feaf2ffd700 1 -- 172.21.15.110:0/1027008740 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feae8009420 con 0x7feaf41371d0 2024-09-15T03:36:44.145 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.119+0000 7feaf2ffd700 1 --2- 172.21.15.110:0/1027008740 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feaf41371d0 0x7feaf4133650 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7feaec00cc40 tx=0x7feaec001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:44.146 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.119+0000 7f70c5562700 1 Processor -- start 2024-09-15T03:36:44.146 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.119+0000 7f70c5562700 1 -- start start 2024-09-15T03:36:44.146 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.119+0000 7f7274ff9700 1 -- 172.21.15.110:0/4217961655 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726c00c7f0 con 0x7f72780acc40 2024-09-15T03:36:44.146 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.119+0000 7f7274ff9700 1 -- 172.21.15.110:0/4217961655 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f726c00c970 con 0x7f72780acc40 2024-09-15T03:36:44.146 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.119+0000 7f727da78700 1 -- 172.21.15.110:0/4217961655 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72781509a0 con 0x7f72780acc40 2024-09-15T03:36:44.146 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.119+0000 7f7274ff9700 1 -- 172.21.15.110:0/4217961655 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726c00cd30 con 0x7f72780acc40 2024-09-15T03:36:44.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.119+0000 7f3fe159a700 1 -- 172.21.15.110:0/3077423658 <== mon.2 v2:172.21.15.110:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd4011630 con 0x7f3fe40aca20 2024-09-15T03:36:44.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.119+0000 7f727da78700 1 -- 172.21.15.110:0/4217961655 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7278138d60 con 0x7f72780acc40 2024-09-15T03:36:44.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.119+0000 7feadb7fe700 1 -- 172.21.15.110:0/1027008740 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaec00a9c0 con 0x7feaf41371d0 2024-09-15T03:36:44.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.119+0000 7feadb7fe700 1 -- 172.21.15.110:0/1027008740 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaec00ab40 con 0x7feaf41371d0 2024-09-15T03:36:44.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.119+0000 7feafa207700 1 -- 172.21.15.110:0/1027008740 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaf413a860 con 0x7feaf41371d0 2024-09-15T03:36:44.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.119+0000 7f3fe159a700 1 -- 172.21.15.110:0/3077423658 <== mon.2 v2:172.21.15.110:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fd40117b0 con 0x7f3fe40aca20 2024-09-15T03:36:44.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.119+0000 7feafa207700 1 -- 172.21.15.110:0/1027008740 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaf415d790 con 0x7feaf41371d0 2024-09-15T03:36:44.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.119+0000 7f70c5562700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f70c00ad0b0 0x7f70c00ab210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:44.149 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.119+0000 7f70c5562700 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70c013b350 con 0x7f70c00ab750 2024-09-15T03:36:44.149 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.119+0000 7f70c5562700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70c00ac5d0 con 0x7f70c00ad480 2024-09-15T03:36:44.149 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.119+0000 7f70c5562700 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70c013e570 con 0x7f70c00ad0b0 2024-09-15T03:36:44.149 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.119+0000 7f3fe8f3b700 1 -- 172.21.15.110:0/3077423658 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f3fe40aca20 msgr2=0x7f3fe40ad390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:44.149 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.119+0000 7f3fe8f3b700 1 --2- 172.21.15.110:0/3077423658 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f3fe40aca20 0x7f3fe40ad390 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3fe40afe40 tx=0x7f3fd40091a0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.149 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.119+0000 7feadb7fe700 1 -- 172.21.15.110:0/1027008740 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaec0070f0 con 0x7feaf41371d0 2024-09-15T03:36:44.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.119+0000 7f644affd700 1 -- 172.21.15.110:0/1563022706 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6464064c70 con 0x7f64640ad120 2024-09-15T03:36:44.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.119+0000 7f64617fa700 1 -- 172.21.15.110:0/1563022706 <== mon.2 v2:172.21.15.110:3301/0 4 ==== mgrmap(e 5) v1 ==== 85847+0+0 (secure 0 0 0) 0x7f645c011be0 con 0x7f64640ad120 2024-09-15T03:36:44.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.123+0000 7f1d10072700 1 -- 172.21.15.110:0/1836684240 wait complete. 2024-09-15T03:36:44.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.123+0000 7f64617fa700 1 --2- 172.21.15.110:0/1563022706 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f644c067c60 0x7f644c06a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:44.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.123+0000 7feadb7fe700 1 -- 172.21.15.110:0/1027008740 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) v1 ==== 85847+0+0 (secure 0 0 0) 0x7feaec007520 con 0x7feaf41371d0 2024-09-15T03:36:44.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.123+0000 7feafa207700 1 -- 172.21.15.110:0/1027008740 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fead40050b0 con 0x7feaf41371d0 2024-09-15T03:36:44.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.123+0000 7f64617fa700 1 -- 172.21.15.110:0/1563022706 <== mon.2 v2:172.21.15.110:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5262+0+0 (secure 0 0 0) 0x7f645c088310 con 0x7f64640ad120 2024-09-15T03:36:44.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.119+0000 7f727da78700 1 -- 172.21.15.110:0/4217961655 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72580050b0 con 0x7f72780acc40 2024-09-15T03:36:44.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.123+0000 7f6468c4d700 1 --2- 172.21.15.110:0/1563022706 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f644c067c60 0x7f644c06a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:44.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.123+0000 7feadb7fe700 1 --2- 172.21.15.110:0/1027008740 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7feae0067c00 0x7feae006a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:44.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.123+0000 7feadb7fe700 1 -- 172.21.15.110:0/1027008740 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5262+0+0 (secure 0 0 0) 0x7feaec0858d0 con 0x7feaf41371d0 2024-09-15T03:36:44.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.123+0000 7feaf37fe700 1 --2- 172.21.15.110:0/1027008740 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7feae0067c00 0x7feae006a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:44.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.123+0000 7f6468c4d700 1 --2- 172.21.15.110:0/1563022706 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f644c067c60 0x7f644c06a110 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f64640ada00 tx=0x7f6458015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:44.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.123+0000 7f7274ff9700 1 -- 172.21.15.110:0/4217961655 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 5) v1 ==== 85847+0+0 (secure 0 0 0) 0x7f726c013680 con 0x7f72780acc40 2024-09-15T03:36:44.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.123+0000 7f1d10072700 1 Processor -- start 2024-09-15T03:36:44.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.119+0000 7f70beffd700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f70c00ad0b0 0x7f70c00ab210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:44.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.119+0000 7f70bf7fe700 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f70c00ab750 0x7f70c0141240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:39978/0 (socket says 172.21.15.110:39978) 2024-09-15T03:36:44.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.119+0000 7f70bf7fe700 1 -- 172.21.15.110:0/2331107455 learned_addr learned my addr 172.21.15.110:0/2331107455 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-15T03:36:44.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.119+0000 7f70beffd700 1 -- 172.21.15.110:0/2331107455 >> v1:172.21.15.177:6789/0 conn(0x7f70c00ad480 legacy=0x7f70c013de60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T03:36:44.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.119+0000 7f70beffd700 1 -- 172.21.15.110:0/2331107455 >> v1:172.21.15.110:6789/0 conn(0x7f70c00ab750 legacy=0x7f70c0141240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:44.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.119+0000 7f70beffd700 1 -- 172.21.15.110:0/2331107455 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70c013e6b0 con 0x7f70c00ad0b0 2024-09-15T03:36:44.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.119+0000 7f70beffd700 1 --2- 172.21.15.110:0/2331107455 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f70c00ad0b0 0x7f70c00ab210 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f70b4009910 tx=0x7f70b40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=995f0cadf8f91d29 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:44.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.119+0000 7f70bdffb700 1 -- 172.21.15.110:0/2331107455 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b400b7e0 con 0x7f70c00ad0b0 2024-09-15T03:36:44.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.119+0000 7f70bdffb700 1 -- 172.21.15.110:0/2331107455 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70b400b940 con 0x7f70c00ad0b0 2024-09-15T03:36:44.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.123+0000 7f70bdffb700 1 -- 172.21.15.110:0/2331107455 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b400bce0 con 0x7f70c00ad0b0 2024-09-15T03:36:44.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.123+0000 7f70c5562700 1 -- 172.21.15.110:0/2331107455 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f70c00ad0b0 msgr2=0x7f70c00ab210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:44.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.123+0000 7f70c5562700 1 --2- 172.21.15.110:0/2331107455 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f70c00ad0b0 0x7f70c00ab210 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f70b4009910 tx=0x7f70b40091a0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.123+0000 7f70c5562700 1 -- 172.21.15.110:0/2331107455 shutdown_connections 2024-09-15T03:36:44.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.123+0000 7f70c5562700 1 --2- 172.21.15.110:0/2331107455 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f70c00ad0b0 0x7f70c00ab210 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.123+0000 7f70c5562700 1 -- 172.21.15.110:0/2331107455 >> 172.21.15.110:0/2331107455 conn(0x7f70c0067670 msgr2=0x7f70c0067a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:44.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.123+0000 7f1d10072700 1 -- start start 2024-09-15T03:36:44.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.123+0000 7f7274ff9700 1 --2- 172.21.15.110:0/4217961655 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f7264067b80 0x7f726406a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:44.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.123+0000 7feaf37fe700 1 --2- 172.21.15.110:0/1027008740 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7feae0067c00 0x7feae006a0b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7feae80098e0 tx=0x7feae8017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:44.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.123+0000 7f727ca76700 1 --2- 172.21.15.110:0/4217961655 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f7264067b80 0x7f726406a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:44.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.123+0000 7f7274ff9700 1 -- 172.21.15.110:0/4217961655 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5262+0+0 (secure 0 0 0) 0x7f726c085600 con 0x7f72780acc40 2024-09-15T03:36:44.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.123+0000 7f1d10072700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d080a9900 0x7f1d0813db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:44.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.123+0000 7f1d10072700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d080ad0f0 0x7f1d0806da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:44.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.123+0000 7f727ca76700 1 --2- 172.21.15.110:0/4217961655 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f7264067b80 0x7f726406a030 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7278137ee0 tx=0x7f7268017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:44.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.123+0000 7f1d10072700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f1d0806df90 0x7f1d0814c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:44.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.123+0000 7f1d10072700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0806cc20 con 0x7f1d080ad0f0 2024-09-15T03:36:44.158 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.123+0000 7f1d10072700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0806d2b0 con 0x7f1d080a9900 2024-09-15T03:36:44.158 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.123+0000 7f1d10072700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0806ce90 con 0x7f1d0806df90 2024-09-15T03:36:44.158 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.123+0000 7f1d0e60f700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f1d0806df90 0x7f1d0814c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:44.158 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.123+0000 7f1d0d60d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d080ad0f0 0x7f1d0806da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:44.158 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.123+0000 7f1d0e60f700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f1d0806df90 0x7f1d0814c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3301/0 says I am v2:172.21.15.110:44870/0 (socket says 172.21.15.110:44870) 2024-09-15T03:36:44.158 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.123+0000 7f1d0e60f700 1 -- 172.21.15.110:0/3895318291 learned_addr learned my addr 172.21.15.110:0/3895318291 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:36:44.158 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.123+0000 7f1d0de0e700 1 --2- 172.21.15.110:0/3895318291 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d080a9900 0x7f1d0813db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:44.158 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.123+0000 7f3fe8f3b700 1 -- 172.21.15.110:0/3077423658 shutdown_connections 2024-09-15T03:36:44.158 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.123+0000 7f3fe8f3b700 1 --2- 172.21.15.110:0/3077423658 >> v2:172.21.15.110:3300/0 conn(0x7f3fe40aa920 0x7f3fe40aada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.123+0000 7f3fe8f3b700 1 --2- 172.21.15.110:0/3077423658 >> v2:172.21.15.177:3300/0 conn(0x7f3fe406bda0 0x7f3fe406e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.123+0000 7f3fe8f3b700 1 --2- 172.21.15.110:0/3077423658 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f3fe40aca20 0x7f3fe40ad390 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.123+0000 7f3fe8f3b700 1 -- 172.21.15.110:0/3077423658 >> 172.21.15.110:0/3077423658 conn(0x7f3fe4067670 msgr2=0x7f3fe4067a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:44.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.123+0000 7f70c5562700 1 -- 172.21.15.110:0/2331107455 shutdown_connections 2024-09-15T03:36:44.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.123+0000 7f70c5562700 1 -- 172.21.15.110:0/2331107455 wait complete. 2024-09-15T03:36:44.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.123+0000 7f1d0d60d700 1 -- 172.21.15.110:0/3895318291 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f1d0806df90 msgr2=0x7f1d0814c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:44.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.123+0000 7f1d0d60d700 1 --2- 172.21.15.110:0/3895318291 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f1d0806df90 0x7f1d0814c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.123+0000 7f1d0d60d700 1 -- 172.21.15.110:0/3895318291 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d080a9900 msgr2=0x7f1d0813db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:44.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.123+0000 7f1d0d60d700 1 --2- 172.21.15.110:0/3895318291 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d080a9900 0x7f1d0813db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.123+0000 7f1d0d60d700 1 -- 172.21.15.110:0/3895318291 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d0400d8d0 con 0x7f1d080ad0f0 2024-09-15T03:36:44.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.123+0000 7f1d0de0e700 1 --2- 172.21.15.110:0/3895318291 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d080a9900 0x7f1d0813db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T03:36:44.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.123+0000 7f1d0d60d700 1 --2- 172.21.15.110:0/3895318291 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d080ad0f0 0x7f1d0806da50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1d0806f6c0 tx=0x7f1cf8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:44.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.123+0000 7f3fe8f3b700 1 -- 172.21.15.110:0/3077423658 shutdown_connections 2024-09-15T03:36:44.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.123+0000 7f3fe8f3b700 1 -- 172.21.15.110:0/3077423658 wait complete. 2024-09-15T03:36:44.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.123+0000 7fdf55443700 1 Processor -- start 2024-09-15T03:36:44.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.123+0000 7fdf55443700 1 -- start start 2024-09-15T03:36:44.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7f70c5562700 1 Processor -- start 2024-09-15T03:36:44.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7f70c5562700 1 -- start start 2024-09-15T03:36:44.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7f3fe8f3b700 1 Processor -- start 2024-09-15T03:36:44.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7fdf55443700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fdf5013da30 0x7fdf5013de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:44.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7fdf55443700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fdf50137140 0x7fdf501375a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:44.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7f3fe8f3b700 1 -- start start 2024-09-15T03:36:44.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7f70c5562700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f70c00ab750 0x7f70c0117d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:44.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7f70c5562700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f70c00ad0b0 0x7f70c0118290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:44.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7f70c5562700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f70c00ad480 0x7f70c0138690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:44.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7f70c5562700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70c01197e0 con 0x7f70c00ab750 2024-09-15T03:36:44.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7f70c5562700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70c0119ac0 con 0x7f70c00ad0b0 2024-09-15T03:36:44.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7f70c5562700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70c00b09b0 con 0x7f70c00ad480 2024-09-15T03:36:44.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7f57f6ac7700 1 Processor -- start 2024-09-15T03:36:44.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7f57f6ac7700 1 -- start start 2024-09-15T03:36:44.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7f70bf7fe700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f70c00ad480 0x7f70c0138690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:44.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7f70beffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f70c00ab750 0x7f70c0117d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:44.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7f70beffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f70c00ab750 0x7f70c0117d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37438/0 (socket says 172.21.15.110:37438) 2024-09-15T03:36:44.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7f70beffd700 1 -- 172.21.15.110:0/2828840836 learned_addr learned my addr 172.21.15.110:0/2828840836 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:36:44.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7f70be7fc700 1 --2- 172.21.15.110:0/2828840836 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f70c00ad0b0 0x7f70c0118290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:44.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7f70beffd700 1 -- 172.21.15.110:0/2828840836 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f70c00ad480 msgr2=0x7f70c0138690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:44.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7f70beffd700 1 --2- 172.21.15.110:0/2828840836 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f70c00ad480 0x7f70c0138690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7f70beffd700 1 -- 172.21.15.110:0/2828840836 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f70c00ad0b0 msgr2=0x7f70c0118290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:44.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7f70beffd700 1 --2- 172.21.15.110:0/2828840836 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f70c00ad0b0 0x7f70c0118290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7f70beffd700 1 -- 172.21.15.110:0/2828840836 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70b4009420 con 0x7f70c00ab750 2024-09-15T03:36:44.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7f70bf7fe700 1 --2- 172.21.15.110:0/2828840836 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f70c00ad480 0x7f70c0138690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T03:36:44.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7f3fe8f3b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3fe406bda0 0x7f3fe413cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:44.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7f70beffd700 1 --2- 172.21.15.110:0/2828840836 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f70c00ab750 0x7f70c0117d50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f70b40030c0 tx=0x7f70b4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:44.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7fdf55443700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf5006e860 con 0x7fdf500ad200 2024-09-15T03:36:44.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7fdf55443700 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf500ad5d0 con 0x7fdf50137140 2024-09-15T03:36:44.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7fdf55443700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf5006dc90 con 0x7fdf5013da30 2024-09-15T03:36:44.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7fdf4e7fc700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fdf5013da30 0x7fdf5013de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:44.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7fdf4e7fc700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fdf5013da30 0x7fdf5013de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.110:49586/0 (socket says 172.21.15.110:49586) 2024-09-15T03:36:44.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7fdf4e7fc700 1 -- 172.21.15.110:0/3328328855 learned_addr learned my addr 172.21.15.110:0/3328328855 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:36:44.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7f3fe8f3b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3fe40aa920 0x7f3fe414bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:44.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7f3fe8f3b700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f3fe40aca20 0x7f3fe4148960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:44.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7f3fe8f3b700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe40a5de0 con 0x7f3fe406bda0 2024-09-15T03:36:44.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7f3fe8f3b700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe414c890 con 0x7f3fe40aa920 2024-09-15T03:36:44.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7f3fe8f3b700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe414cd20 con 0x7f3fe40aca20 2024-09-15T03:36:44.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7f3fe259c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3fe406bda0 0x7f3fe413cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:44.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7f3fe2d9d700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f3fe40aca20 0x7f3fe4148960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:44.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7f3fe259c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3fe406bda0 0x7f3fe413cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37442/0 (socket says 172.21.15.110:37442) 2024-09-15T03:36:44.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7f3fe259c700 1 -- 172.21.15.110:0/3196407668 learned_addr learned my addr 172.21.15.110:0/3196407668 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:36:44.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7fdf4e7fc700 1 -- 172.21.15.110:0/3328328855 >> v2:172.21.15.110:3300/0 conn(0x7fdf50137140 msgr2=0x7fdf501375a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:36:44.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7f3fe1d9b700 1 --2- 172.21.15.110:0/3196407668 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3fe40aa920 0x7f3fe414bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:44.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7f3fe2d9d700 1 -- 172.21.15.110:0/3196407668 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3fe40aa920 msgr2=0x7f3fe414bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:44.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7f3fe2d9d700 1 --2- 172.21.15.110:0/3196407668 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3fe40aa920 0x7f3fe414bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7f3fe2d9d700 1 -- 172.21.15.110:0/3196407668 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3fe406bda0 msgr2=0x7f3fe413cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:44.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7f3fe2d9d700 1 --2- 172.21.15.110:0/3196407668 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3fe406bda0 0x7f3fe413cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7f3fe2d9d700 1 -- 172.21.15.110:0/3196407668 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fdc009a20 con 0x7f3fe40aca20 2024-09-15T03:36:44.169 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7f3fe1d9b700 1 --2- 172.21.15.110:0/3196407668 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3fe40aa920 0x7f3fe414bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T03:36:44.169 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7f57f6ac7700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f57f00ad280 0x7f57f0140db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:44.169 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7f57f6ac7700 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57f00ab050 con 0x7f57f0141720 2024-09-15T03:36:44.169 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7f57f6ac7700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57f00ad650 con 0x7f57f01412f0 2024-09-15T03:36:44.169 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7f57f6ac7700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57f00a9700 con 0x7f57f00ad280 2024-09-15T03:36:44.169 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7f57f5064700 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f57f0141720 0x7f57f00a8ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:39990/0 (socket says 172.21.15.110:39990) 2024-09-15T03:36:44.169 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7f57f5064700 1 -- 172.21.15.110:0/1492116477 learned_addr learned my addr 172.21.15.110:0/1492116477 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-15T03:36:44.169 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7f57f4863700 1 --2- 172.21.15.110:0/1492116477 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f57f00ad280 0x7f57f0140db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:44.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7fdf4f7fe700 1 --2- 172.21.15.110:0/3328328855 >> v2:172.21.15.110:3300/0 conn(0x7fdf50137140 0x7fdf501375a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:44.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7fdf4e7fc700 1 --2- 172.21.15.110:0/3328328855 >> v2:172.21.15.110:3300/0 conn(0x7fdf50137140 0x7fdf501375a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7fdf4e7fc700 1 -- 172.21.15.110:0/3328328855 >> v1:172.21.15.177:6789/0 conn(0x7fdf500ad200 legacy=0x7fdf5006cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:44.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7fdf4e7fc700 1 -- 172.21.15.110:0/3328328855 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf5006ead0 con 0x7fdf5013da30 2024-09-15T03:36:44.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7fdf4f7fe700 1 --2- 172.21.15.110:0/3328328855 >> v2:172.21.15.110:3300/0 conn(0x7fdf50137140 0x7fdf501375a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T03:36:44.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7f3fe2d9d700 1 --2- 172.21.15.110:0/3196407668 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f3fe40aca20 0x7f3fe4148960 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc009f40 tx=0x7f3fdc00b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:44.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7f57f4863700 1 -- 172.21.15.110:0/1492116477 >> v1:172.21.15.177:6789/0 conn(0x7f57f01412f0 legacy=0x7f57f00a88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:44.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7f57f4863700 1 -- 172.21.15.110:0/1492116477 >> v1:172.21.15.110:6789/0 conn(0x7f57f0141720 legacy=0x7f57f00a8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:44.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7f57f4863700 1 -- 172.21.15.110:0/1492116477 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f00ac2b0 con 0x7f57f00ad280 2024-09-15T03:36:44.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7f57f4863700 1 --2- 172.21.15.110:0/1492116477 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f57f00ad280 0x7f57f0140db0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f57e0014500 tx=0x7f57e000f9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=996906ae16c96133 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:44.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7fdf4e7fc700 1 --2- 172.21.15.110:0/3328328855 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdf5013da30 0x7fdf5013de90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdf500ada70 tx=0x7fdf44009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f96a8fa1a0513cce server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:44.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7f57ef7fe700 1 -- 172.21.15.110:0/1492116477 <== mon.2 v2:172.21.15.110:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e0015700 con 0x7f57f00ad280 2024-09-15T03:36:44.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7f3fd27fc700 1 -- 172.21.15.110:0/3196407668 <== mon.2 v2:172.21.15.110:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc00cc20 con 0x7f3fe40aca20 2024-09-15T03:36:44.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7f57ef7fe700 1 -- 172.21.15.110:0/1492116477 <== mon.2 v2:172.21.15.110:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57e0015860 con 0x7f57f00ad280 2024-09-15T03:36:44.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7f57ef7fe700 1 -- 172.21.15.110:0/1492116477 <== mon.2 v2:172.21.15.110:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e0015c00 con 0x7f57f00ad280 2024-09-15T03:36:44.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7f3fd27fc700 1 -- 172.21.15.110:0/3196407668 <== mon.2 v2:172.21.15.110:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fdc00f040 con 0x7f3fe40aca20 2024-09-15T03:36:44.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7f64617fa700 1 -- 172.21.15.110:0/1563022706 <== mon.2 v2:172.21.15.110:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f645c0555b0 con 0x7f64640ad120 2024-09-15T03:36:44.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7f3fe8f3b700 1 -- 172.21.15.110:0/3196407668 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fd4009420 con 0x7f3fe40aca20 2024-09-15T03:36:44.174 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7f3fd27fc700 1 -- 172.21.15.110:0/3196407668 <== mon.2 v2:172.21.15.110:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc014480 con 0x7f3fe40aca20 2024-09-15T03:36:44.174 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.127+0000 7f3fe8f3b700 1 -- 172.21.15.110:0/3196407668 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fe4148ea0 con 0x7f3fe40aca20 2024-09-15T03:36:44.174 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.131+0000 7f57f6ac7700 1 -- 172.21.15.110:0/1492116477 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f57f00ad280 msgr2=0x7f57f0140db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:44.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.131+0000 7f57f6ac7700 1 --2- 172.21.15.110:0/1492116477 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f57f00ad280 0x7f57f0140db0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f57e0014500 tx=0x7f57e000f9c0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.131+0000 7f57f6ac7700 1 -- 172.21.15.110:0/1492116477 shutdown_connections 2024-09-15T03:36:44.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.131+0000 7f57f6ac7700 1 --2- 172.21.15.110:0/1492116477 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f57f00ad280 0x7f57f0140db0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.131+0000 7feadb7fe700 1 -- 172.21.15.110:0/1027008740 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7feaec052bf0 con 0x7feaf41371d0 2024-09-15T03:36:44.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.131+0000 7f1cfdffb700 1 -- 172.21.15.110:0/3895318291 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf801f070 con 0x7f1d080ad0f0 2024-09-15T03:36:44.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.131+0000 7f1cfdffb700 1 -- 172.21.15.110:0/3895318291 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cf8000bc0 con 0x7f1d080ad0f0 2024-09-15T03:36:44.175 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.131+0000 7f1cfdffb700 1 -- 172.21.15.110:0/3895318291 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf8025420 con 0x7f1d080ad0f0 2024-09-15T03:36:44.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.131+0000 7f1d10072700 1 -- 172.21.15.110:0/3895318291 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cf80094f0 con 0x7f1d080ad0f0 2024-09-15T03:36:44.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.131+0000 7f1d10072700 1 -- 172.21.15.110:0/3895318291 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d0806bac0 con 0x7f1d080ad0f0 2024-09-15T03:36:44.176 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.131+0000 7f57f6ac7700 1 -- 172.21.15.110:0/1492116477 >> 172.21.15.110:0/1492116477 conn(0x7f57f0067a10 msgr2=0x7f57f0067e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:44.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.131+0000 7f70a6ffd700 1 -- 172.21.15.110:0/2828840836 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b401b680 con 0x7f70c00ab750 2024-09-15T03:36:44.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.131+0000 7f70c5562700 1 -- 172.21.15.110:0/2828840836 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70c0138bd0 con 0x7f70c00ab750 2024-09-15T03:36:44.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.131+0000 7f70c5562700 1 -- 172.21.15.110:0/2828840836 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70c0139040 con 0x7f70c00ab750 2024-09-15T03:36:44.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.131+0000 7f70a6ffd700 1 -- 172.21.15.110:0/2828840836 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70b401b7e0 con 0x7f70c00ab750 2024-09-15T03:36:44.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.131+0000 7f70a6ffd700 1 -- 172.21.15.110:0/2828840836 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b401bbc0 con 0x7f70c00ab750 2024-09-15T03:36:44.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.131+0000 7f1d10072700 1 -- 172.21.15.110:0/3895318291 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d08064c70 con 0x7f1d080ad0f0 2024-09-15T03:36:44.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.131+0000 7f3fe8f3b700 1 -- 172.21.15.110:0/3196407668 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fe414ca00 con 0x7f3fe40aca20 2024-09-15T03:36:44.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.131+0000 7f3fd27fc700 1 -- 172.21.15.110:0/3196407668 <== mon.2 v2:172.21.15.110:3301/0 4 ==== mgrmap(e 5) v1 ==== 85847+0+0 (secure 0 0 0) 0x7f3fdc003020 con 0x7f3fe40aca20 2024-09-15T03:36:44.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.131+0000 7f3fd27fc700 1 --2- 172.21.15.110:0/3196407668 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f3fc4067b50 0x7f3fc406a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:44.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.131+0000 7f3fd27fc700 1 -- 172.21.15.110:0/3196407668 <== mon.2 v2:172.21.15.110:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5262+0+0 (secure 0 0 0) 0x7f3fdc085fb0 con 0x7f3fe40aca20 2024-09-15T03:36:44.178 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.131+0000 7f70c5562700 1 -- 172.21.15.110:0/2828840836 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70a00050b0 con 0x7f70c00ab750 2024-09-15T03:36:44.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.131+0000 7f57f6ac7700 1 -- 172.21.15.110:0/1492116477 shutdown_connections 2024-09-15T03:36:44.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.131+0000 7f57f6ac7700 1 -- 172.21.15.110:0/1492116477 wait complete. 2024-09-15T03:36:44.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.131+0000 7f70a6ffd700 1 -- 172.21.15.110:0/2828840836 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) v1 ==== 85847+0+0 (secure 0 0 0) 0x7f70b401c450 con 0x7f70c00ab750 2024-09-15T03:36:44.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.131+0000 7f70a6ffd700 1 --2- 172.21.15.110:0/2828840836 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f70ac067a80 0x7f70ac069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:44.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.131+0000 7f70a6ffd700 1 -- 172.21.15.110:0/2828840836 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5262+0+0 (secure 0 0 0) 0x7f70b40874e0 con 0x7f70c00ab750 2024-09-15T03:36:44.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.131+0000 7f70be7fc700 1 --2- 172.21.15.110:0/2828840836 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f70ac067a80 0x7f70ac069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:44.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.135+0000 7f1cfdffb700 1 -- 172.21.15.110:0/3895318291 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) v1 ==== 85847+0+0 (secure 0 0 0) 0x7f1cf8023070 con 0x7f1d080ad0f0 2024-09-15T03:36:44.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.135+0000 7f1cfdffb700 1 --2- 172.21.15.110:0/3895318291 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f1cec067b00 0x7f1cec069fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:44.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.135+0000 7f1cfdffb700 1 -- 172.21.15.110:0/3895318291 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5262+0+0 (secure 0 0 0) 0x7f1cf8086410 con 0x7f1d080ad0f0 2024-09-15T03:36:44.181 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.135+0000 7f70be7fc700 1 --2- 172.21.15.110:0/2828840836 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f70ac067a80 0x7f70ac069f30 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f70c0119150 tx=0x7f70b0009060 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:44.181 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.135+0000 7f3fe259c700 1 --2- 172.21.15.110:0/3196407668 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f3fc4067b50 0x7f3fc406a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:44.181 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.135+0000 7f57f6ac7700 1 Processor -- start 2024-09-15T03:36:44.181 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.135+0000 7f57f6ac7700 1 -- start start 2024-09-15T03:36:44.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.135+0000 7f1d0de0e700 1 --2- 172.21.15.110:0/3895318291 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f1cec067b00 0x7f1cec069fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:44.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.135+0000 7f57f6ac7700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f57f00ad280 0x7f57f0140410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:44.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.135+0000 7f57f6ac7700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f57f01412f0 0x7f57f013dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:44.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.135+0000 7f3fe259c700 1 --2- 172.21.15.110:0/3196407668 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f3fc4067b50 0x7f3fc406a000 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3fd40093f0 tx=0x7f3fd4017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:44.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.135+0000 7f57f6ac7700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f57f0141720 0x7f57f012c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:44.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.135+0000 7f57f6ac7700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f006d5a0 con 0x7f57f00ad280 2024-09-15T03:36:44.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.135+0000 7f57f6ac7700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f013c9e0 con 0x7f57f01412f0 2024-09-15T03:36:44.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.135+0000 7f57f6ac7700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57f0086d80 con 0x7f57f0141720 2024-09-15T03:36:44.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.135+0000 7f57effff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f57f01412f0 0x7f57f013dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:44.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.135+0000 7f57f5064700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f57f0141720 0x7f57f012c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:44.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.135+0000 7f57effff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f57f01412f0 0x7f57f013dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.110:49620/0 (socket says 172.21.15.110:49620) 2024-09-15T03:36:44.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.135+0000 7f57effff700 1 -- 172.21.15.110:0/1289558709 learned_addr learned my addr 172.21.15.110:0/1289558709 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:36:44.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.135+0000 7f7274ff9700 1 -- 172.21.15.110:0/4217961655 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f726c052940 con 0x7f72780acc40 2024-09-15T03:36:44.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.135+0000 7fdf4dffb700 1 -- 172.21.15.110:0/3328328855 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4400b860 con 0x7fdf5013da30 2024-09-15T03:36:44.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.135+0000 7fdf4dffb700 1 -- 172.21.15.110:0/3328328855 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf4400b9c0 con 0x7fdf5013da30 2024-09-15T03:36:44.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.135+0000 7f1d0de0e700 1 --2- 172.21.15.110:0/3895318291 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f1cec067b00 0x7f1cec069fb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1d0400d440 tx=0x7f1d0400f040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:44.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.135+0000 7fdf4dffb700 1 -- 172.21.15.110:0/3328328855 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4400bd60 con 0x7fdf5013da30 2024-09-15T03:36:44.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.135+0000 7f57f5064700 1 -- 172.21.15.110:0/1289558709 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f57f01412f0 msgr2=0x7f57f013dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:44.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.135+0000 7f57f5064700 1 --2- 172.21.15.110:0/1289558709 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f57f01412f0 0x7f57f013dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.135+0000 7f57f4863700 1 --2- 172.21.15.110:0/1289558709 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f57f00ad280 0x7f57f0140410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:44.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.135+0000 7f57f5064700 1 -- 172.21.15.110:0/1289558709 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f57f00ad280 msgr2=0x7f57f0140410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:44.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.135+0000 7fdf55443700 1 -- 172.21.15.110:0/3328328855 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdf5013da30 msgr2=0x7fdf5013de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:44.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.135+0000 7fdf55443700 1 --2- 172.21.15.110:0/3328328855 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdf5013da30 0x7fdf5013de90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdf500ada70 tx=0x7fdf44009250 comp rx=0 tx=0).stop 2024-09-15T03:36:44.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.135+0000 7f57f5064700 1 --2- 172.21.15.110:0/1289558709 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f57f00ad280 0x7f57f0140410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.135+0000 7f57f5064700 1 -- 172.21.15.110:0/1289558709 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e000fc60 con 0x7f57f0141720 2024-09-15T03:36:44.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.135+0000 7f57effff700 1 --2- 172.21.15.110:0/1289558709 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f57f01412f0 0x7f57f013dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T03:36:44.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.135+0000 7f57f4863700 1 --2- 172.21.15.110:0/1289558709 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f57f00ad280 0x7f57f0140410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T03:36:44.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.135+0000 7f57f5064700 1 --2- 172.21.15.110:0/1289558709 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f57f0141720 0x7f57f012c680 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f57e800cb40 tx=0x7f57e8001900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:44.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.135+0000 7fdf55443700 1 -- 172.21.15.110:0/3328328855 shutdown_connections 2024-09-15T03:36:44.188 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.135+0000 7fdf55443700 1 --2- 172.21.15.110:0/3328328855 >> v2:172.21.15.110:3300/0 conn(0x7fdf50137140 0x7fdf501375a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.188 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.135+0000 7fdf55443700 1 --2- 172.21.15.110:0/3328328855 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdf5013da30 0x7fdf5013de90 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.188 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.135+0000 7fdf55443700 1 -- 172.21.15.110:0/3328328855 >> 172.21.15.110:0/3328328855 conn(0x7fdf50067a10 msgr2=0x7fdf50067e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:44.188 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.139+0000 7fdf55443700 1 -- 172.21.15.110:0/3328328855 shutdown_connections 2024-09-15T03:36:44.188 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.139+0000 7f57ecff9700 1 -- 172.21.15.110:0/1289558709 <== mon.2 v2:172.21.15.110:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e800a940 con 0x7f57f0141720 2024-09-15T03:36:44.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.139+0000 7f3fd27fc700 1 -- 172.21.15.110:0/3196407668 <== mon.2 v2:172.21.15.110:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f3fdc053250 con 0x7f3fe40aca20 2024-09-15T03:36:44.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.139+0000 7f57f6ac7700 1 -- 172.21.15.110:0/1289558709 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57f006c140 con 0x7f57f0141720 2024-09-15T03:36:44.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.139+0000 7f57f6ac7700 1 -- 172.21.15.110:0/1289558709 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57f0086b10 con 0x7f57f0141720 2024-09-15T03:36:44.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.139+0000 7f57ecff9700 1 -- 172.21.15.110:0/1289558709 <== mon.2 v2:172.21.15.110:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57e800aaa0 con 0x7f57f0141720 2024-09-15T03:36:44.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.139+0000 7f57ecff9700 1 -- 172.21.15.110:0/1289558709 <== mon.2 v2:172.21.15.110:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e800ae40 con 0x7f57f0141720 2024-09-15T03:36:44.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.139+0000 7f57ecff9700 1 -- 172.21.15.110:0/1289558709 <== mon.2 v2:172.21.15.110:3301/0 4 ==== mgrmap(e 5) v1 ==== 85847+0+0 (secure 0 0 0) 0x7f57e8007300 con 0x7f57f0141720 2024-09-15T03:36:44.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.143+0000 7f70a6ffd700 1 -- 172.21.15.110:0/2828840836 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f70b40547e0 con 0x7f70c00ab750 2024-09-15T03:36:44.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.143+0000 7fdf55443700 1 -- 172.21.15.110:0/3328328855 wait complete. 2024-09-15T03:36:44.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.143+0000 7f57ecff9700 1 --2- 172.21.15.110:0/1289558709 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f57dc067ad0 0x7f57dc069f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:44.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.143+0000 7f57f4863700 1 --2- 172.21.15.110:0/1289558709 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f57dc067ad0 0x7f57dc069f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:44.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.143+0000 7f57ecff9700 1 -- 172.21.15.110:0/1289558709 <== mon.2 v2:172.21.15.110:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5262+0+0 (secure 0 0 0) 0x7f57e80856d0 con 0x7f57f0141720 2024-09-15T03:36:44.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.143+0000 7f57f4863700 1 --2- 172.21.15.110:0/1289558709 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f57dc067ad0 0x7f57dc069f80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f57e0003080 tx=0x7f57e001f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:44.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.143+0000 7fdf55443700 1 Processor -- start 2024-09-15T03:36:44.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.143+0000 7fdf55443700 1 -- start start 2024-09-15T03:36:44.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.143+0000 7fdf55443700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdf500ad200 0x7fdf500af0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:44.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.143+0000 7fdf55443700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fdf50137140 0x7fdf50151750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:44.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.143+0000 7fdf55443700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf5013da30 0x7fdf50151c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:44.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.143+0000 7fdf55443700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5013ac10 con 0x7fdf5013da30 2024-09-15T03:36:44.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.143+0000 7fdf55443700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf500a5310 con 0x7fdf500ad200 2024-09-15T03:36:44.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.143+0000 7fdf55443700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5013a8a0 con 0x7fdf50137140 2024-09-15T03:36:44.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.143+0000 7fdf4f7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf5013da30 0x7fdf50151c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:44.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.143+0000 7fdf4e7fc700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fdf50137140 0x7fdf50151750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:44.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.143+0000 7fdf4f7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf5013da30 0x7fdf50151c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37458/0 (socket says 172.21.15.110:37458) 2024-09-15T03:36:44.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.143+0000 7fdf4e7fc700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fdf50137140 0x7fdf50151750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3301/0 says I am v2:172.21.15.110:44910/0 (socket says 172.21.15.110:44910) 2024-09-15T03:36:44.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.143+0000 7fdf4f7fe700 1 -- 172.21.15.110:0/1758396172 learned_addr learned my addr 172.21.15.110:0/1758396172 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:36:44.194 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.143+0000 7fdf4effd700 1 --2- 172.21.15.110:0/1758396172 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdf500ad200 0x7fdf500af0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:44.194 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.143+0000 7fdf4f7fe700 1 -- 172.21.15.110:0/1758396172 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fdf50137140 msgr2=0x7fdf50151750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:44.194 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.143+0000 7fdf4f7fe700 1 --2- 172.21.15.110:0/1758396172 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fdf50137140 0x7fdf50151750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.194 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.143+0000 7fdf4f7fe700 1 -- 172.21.15.110:0/1758396172 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdf500ad200 msgr2=0x7fdf500af0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:44.194 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.143+0000 7fdf4f7fe700 1 --2- 172.21.15.110:0/1758396172 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdf500ad200 0x7fdf500af0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.194 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.143+0000 7fdf4f7fe700 1 -- 172.21.15.110:0/1758396172 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf440094f0 con 0x7fdf5013da30 2024-09-15T03:36:44.194 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.143+0000 7fdf4e7fc700 1 --2- 172.21.15.110:0/1758396172 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fdf50137140 0x7fdf50151750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T03:36:44.194 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.147+0000 7fdf4f7fe700 1 --2- 172.21.15.110:0/1758396172 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf5013da30 0x7fdf50151c90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdf4800c880 tx=0x7fdf48008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:44.194 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.151+0000 7f57f6ac7700 1 -- 172.21.15.110:0/1289558709 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57d00050b0 con 0x7f57f0141720 2024-09-15T03:36:44.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.151+0000 7fdf36ffd700 1 -- 172.21.15.110:0/1758396172 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf48011600 con 0x7fdf5013da30 2024-09-15T03:36:44.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.151+0000 7fdf36ffd700 1 -- 172.21.15.110:0/1758396172 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf48011760 con 0x7fdf5013da30 2024-09-15T03:36:44.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.151+0000 7fdf36ffd700 1 -- 172.21.15.110:0/1758396172 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf48011b20 con 0x7fdf5013da30 2024-09-15T03:36:44.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.155+0000 7fdf55443700 1 -- 172.21.15.110:0/1758396172 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf500afc30 con 0x7fdf5013da30 2024-09-15T03:36:44.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.155+0000 7fdf55443700 1 -- 172.21.15.110:0/1758396172 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf5013b2c0 con 0x7fdf5013da30 2024-09-15T03:36:44.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.155+0000 7f1cfdffb700 1 -- 172.21.15.110:0/3895318291 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f1cf8053630 con 0x7f1d080ad0f0 2024-09-15T03:36:44.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.155+0000 7fdf36ffd700 1 -- 172.21.15.110:0/1758396172 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) v1 ==== 85847+0+0 (secure 0 0 0) 0x7fdf4801a400 con 0x7fdf5013da30 2024-09-15T03:36:44.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.159+0000 7fdf36ffd700 1 --2- 172.21.15.110:0/1758396172 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fdf3c067a80 0x7fdf3c069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:44.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.159+0000 7fdf36ffd700 1 -- 172.21.15.110:0/1758396172 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5262+0+0 (secure 0 0 0) 0x7fdf480858a0 con 0x7fdf5013da30 2024-09-15T03:36:44.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.159+0000 7fdf4effd700 1 --2- 172.21.15.110:0/1758396172 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fdf3c067a80 0x7fdf3c069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:44.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.159+0000 7fdf4effd700 1 --2- 172.21.15.110:0/1758396172 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fdf3c067a80 0x7fdf3c069f30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdf40007fd0 tx=0x7fdf400016f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:44.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.159+0000 7f57ecff9700 1 -- 172.21.15.110:0/1289558709 <== mon.2 v2:172.21.15.110:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f57e8052a60 con 0x7f57f0141720 2024-09-15T03:36:44.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.163+0000 7fdf55443700 1 -- 172.21.15.110:0/1758396172 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf300050b0 con 0x7fdf5013da30 2024-09-15T03:36:44.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.171+0000 7fdf36ffd700 1 -- 172.21.15.110:0/1758396172 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fdf480523d0 con 0x7fdf5013da30 2024-09-15T03:36:44.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.427+0000 7f644affd700 1 -- 172.21.15.110:0/1563022706 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f646406c500 con 0x7f64640ad120 2024-09-15T03:36:44.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.427+0000 7f64617fa700 1 -- 172.21.15.110:0/1563022706 <== mon.2 v2:172.21.15.110:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f645c058630 con 0x7f64640ad120 2024-09-15T03:36:44.434 INFO:teuthology.orchestra.run.smithi110.stdout:60129542145 2024-09-15T03:36:44.438 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.435+0000 7feafa207700 1 -- 172.21.15.110:0/1027008740 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fead4005360 con 0x7feaf41371d0 2024-09-15T03:36:44.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.435+0000 7f646b6b2700 1 -- 172.21.15.110:0/1563022706 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f644c067c60 msgr2=0x7f644c06a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:44.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.435+0000 7f646b6b2700 1 --2- 172.21.15.110:0/1563022706 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f644c067c60 0x7f644c06a110 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f64640ada00 tx=0x7f6458015000 comp rx=0 tx=0).stop 2024-09-15T03:36:44.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.435+0000 7f646b6b2700 1 -- 172.21.15.110:0/1563022706 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f64640ad120 msgr2=0x7f64640af0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:44.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.435+0000 7f646b6b2700 1 --2- 172.21.15.110:0/1563022706 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f64640ad120 0x7f64640af0e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f645c00c2a0 tx=0x7f645c001b70 comp rx=0 tx=0).stop 2024-09-15T03:36:44.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.435+0000 7f646b6b2700 1 -- 172.21.15.110:0/1563022706 shutdown_connections 2024-09-15T03:36:44.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.435+0000 7f646b6b2700 1 --2- 172.21.15.110:0/1563022706 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f644c067c60 0x7f644c06a110 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.435+0000 7f646b6b2700 1 --2- 172.21.15.110:0/1563022706 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64640a4d60 0x7f646414d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.435+0000 7f646b6b2700 1 --2- 172.21.15.110:0/1563022706 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f646413a390 0x7f646413c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.435+0000 7f646b6b2700 1 --2- 172.21.15.110:0/1563022706 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f64640ad120 0x7f64640af0e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.435+0000 7f646b6b2700 1 -- 172.21.15.110:0/1563022706 >> 172.21.15.110:0/1563022706 conn(0x7f6464067a10 msgr2=0x7f64641342b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:44.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.435+0000 7feadb7fe700 1 -- 172.21.15.110:0/1027008740 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7feaec055c70 con 0x7feaf41371d0 2024-09-15T03:36:44.441 INFO:teuthology.orchestra.run.smithi110.stdout:60129542145 2024-09-15T03:36:44.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.435+0000 7f727da78700 1 -- 172.21.15.110:0/4217961655 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7258005630 con 0x7f72780acc40 2024-09-15T03:36:44.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.435+0000 7f7274ff9700 1 -- 172.21.15.110:0/4217961655 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f726c0559c0 con 0x7f72780acc40 2024-09-15T03:36:44.442 INFO:teuthology.orchestra.run.smithi110.stdout:55834574850 2024-09-15T03:36:44.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.435+0000 7f646b6b2700 1 -- 172.21.15.110:0/1563022706 shutdown_connections 2024-09-15T03:36:44.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.435+0000 7f646b6b2700 1 -- 172.21.15.110:0/1563022706 wait complete. 2024-09-15T03:36:44.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.435+0000 7fead97fa700 1 -- 172.21.15.110:0/1027008740 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7feae0067c00 msgr2=0x7feae006a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:44.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.435+0000 7fead97fa700 1 --2- 172.21.15.110:0/1027008740 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7feae0067c00 0x7feae006a0b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7feae80098e0 tx=0x7feae8017000 comp rx=0 tx=0).stop 2024-09-15T03:36:44.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.439+0000 7fead97fa700 1 -- 172.21.15.110:0/1027008740 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feaf41371d0 msgr2=0x7feaf4133650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:44.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.439+0000 7fead97fa700 1 --2- 172.21.15.110:0/1027008740 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feaf41371d0 0x7feaf4133650 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7feaec00cc40 tx=0x7feaec001a70 comp rx=0 tx=0).stop 2024-09-15T03:36:44.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.439+0000 7fead97fa700 1 -- 172.21.15.110:0/1027008740 shutdown_connections 2024-09-15T03:36:44.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.439+0000 7fead97fa700 1 --2- 172.21.15.110:0/1027008740 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7feae0067c00 0x7feae006a0b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.439+0000 7fead97fa700 1 --2- 172.21.15.110:0/1027008740 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7feaf413da10 0x7feaf415d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.439+0000 7fead97fa700 1 --2- 172.21.15.110:0/1027008740 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feaf41371d0 0x7feaf4133650 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.439+0000 7fead97fa700 1 --2- 172.21.15.110:0/1027008740 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feaf40ad390 0x7feaf4132d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.439+0000 7fead97fa700 1 -- 172.21.15.110:0/1027008740 >> 172.21.15.110:0/1027008740 conn(0x7feaf4067670 msgr2=0x7feaf40b0610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:44.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.439+0000 7fead97fa700 1 -- 172.21.15.110:0/1027008740 shutdown_connections 2024-09-15T03:36:44.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.439+0000 7fead97fa700 1 -- 172.21.15.110:0/1027008740 wait complete. 2024-09-15T03:36:44.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.447+0000 7f727da78700 1 -- 172.21.15.110:0/4217961655 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f7264067b80 msgr2=0x7f726406a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:44.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.447+0000 7f727da78700 1 --2- 172.21.15.110:0/4217961655 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f7264067b80 0x7f726406a030 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7278137ee0 tx=0x7f7268017000 comp rx=0 tx=0).stop 2024-09-15T03:36:44.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.447+0000 7f727da78700 1 -- 172.21.15.110:0/4217961655 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f72780acc40 msgr2=0x7f727813a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:44.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.447+0000 7f727da78700 1 --2- 172.21.15.110:0/4217961655 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f72780acc40 0x7f727813a6d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f726c00ea50 tx=0x7f726c0091a0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.447+0000 7f727da78700 1 -- 172.21.15.110:0/4217961655 shutdown_connections 2024-09-15T03:36:44.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.447+0000 7f727da78700 1 --2- 172.21.15.110:0/4217961655 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f7264067b80 0x7f726406a030 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.447+0000 7f727da78700 1 --2- 172.21.15.110:0/4217961655 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72780ad010 0x7f7278138820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.447+0000 7f727da78700 1 --2- 172.21.15.110:0/4217961655 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f72780acc40 0x7f727813a6d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.447+0000 7f727da78700 1 --2- 172.21.15.110:0/4217961655 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f727806eba0 0x7f7278144810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.447+0000 7f727da78700 1 -- 172.21.15.110:0/4217961655 >> 172.21.15.110:0/4217961655 conn(0x7f7278067670 msgr2=0x7f727813e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:44.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.447+0000 7f727da78700 1 -- 172.21.15.110:0/4217961655 shutdown_connections 2024-09-15T03:36:44.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.447+0000 7f727da78700 1 -- 172.21.15.110:0/4217961655 wait complete. 2024-09-15T03:36:44.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.447+0000 7f70c5562700 1 -- 172.21.15.110:0/2828840836 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f70a0005630 con 0x7f70c00ab750 2024-09-15T03:36:44.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.451+0000 7f3fe8f3b700 1 -- 172.21.15.110:0/3196407668 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f3fe414c650 con 0x7f3fe40aca20 2024-09-15T03:36:44.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.451+0000 7f70a6ffd700 1 -- 172.21.15.110:0/2828840836 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f70b4057860 con 0x7f70c00ab750 2024-09-15T03:36:44.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.451+0000 7f3fd27fc700 1 -- 172.21.15.110:0/3196407668 <== mon.2 v2:172.21.15.110:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3fdc0562d0 con 0x7f3fe40aca20 2024-09-15T03:36:44.455 INFO:teuthology.orchestra.run.smithi110.stdout:60129542146 2024-09-15T03:36:44.457 INFO:teuthology.orchestra.run.smithi110.stdout:60129542146 2024-09-15T03:36:44.458 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542147 got 60129542145 for osd.5 2024-09-15T03:36:44.459 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542147 got 60129542145 for osd.3 2024-09-15T03:36:44.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.451+0000 7f3fc3fff700 1 -- 172.21.15.110:0/3196407668 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f3fc4067b50 msgr2=0x7f3fc406a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:44.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.451+0000 7f3fc3fff700 1 --2- 172.21.15.110:0/3196407668 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f3fc4067b50 0x7f3fc406a000 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3fd40093f0 tx=0x7f3fd4017000 comp rx=0 tx=0).stop 2024-09-15T03:36:44.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.451+0000 7f3fc3fff700 1 -- 172.21.15.110:0/3196407668 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f3fe40aca20 msgr2=0x7f3fe4148960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:44.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.455+0000 7f3fc3fff700 1 --2- 172.21.15.110:0/3196407668 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f3fe40aca20 0x7f3fe4148960 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc009f40 tx=0x7f3fdc00b040 comp rx=0 tx=0).stop 2024-09-15T03:36:44.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.455+0000 7f3fc3fff700 1 -- 172.21.15.110:0/3196407668 shutdown_connections 2024-09-15T03:36:44.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.455+0000 7f3fc3fff700 1 --2- 172.21.15.110:0/3196407668 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f3fc4067b50 0x7f3fc406a000 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.455+0000 7f3fc3fff700 1 --2- 172.21.15.110:0/3196407668 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f3fe40aca20 0x7f3fe4148960 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.455+0000 7f3fc3fff700 1 --2- 172.21.15.110:0/3196407668 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3fe40aa920 0x7f3fe414bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.455+0000 7f3fc3fff700 1 --2- 172.21.15.110:0/3196407668 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3fe406bda0 0x7f3fe413cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.455+0000 7f3fc3fff700 1 -- 172.21.15.110:0/3196407668 >> 172.21.15.110:0/3196407668 conn(0x7f3fe4067670 msgr2=0x7f3fe4134010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:44.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.455+0000 7f70c5562700 1 -- 172.21.15.110:0/2828840836 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f70ac067a80 msgr2=0x7f70ac069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:44.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.455+0000 7f70c5562700 1 --2- 172.21.15.110:0/2828840836 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f70ac067a80 0x7f70ac069f30 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f70c0119150 tx=0x7f70b0009060 comp rx=0 tx=0).stop 2024-09-15T03:36:44.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.455+0000 7f70c5562700 1 -- 172.21.15.110:0/2828840836 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f70c00ab750 msgr2=0x7f70c0117d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:44.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.455+0000 7f70c5562700 1 --2- 172.21.15.110:0/2828840836 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f70c00ab750 0x7f70c0117d50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f70b40030c0 tx=0x7f70b4016000 comp rx=0 tx=0).stop 2024-09-15T03:36:44.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.455+0000 7f70c5562700 1 -- 172.21.15.110:0/2828840836 shutdown_connections 2024-09-15T03:36:44.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.455+0000 7f70c5562700 1 --2- 172.21.15.110:0/2828840836 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f70ac067a80 0x7f70ac069f30 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.455+0000 7f70c5562700 1 --2- 172.21.15.110:0/2828840836 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f70c00ad480 0x7f70c0138690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.455+0000 7f70c5562700 1 --2- 172.21.15.110:0/2828840836 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f70c00ad0b0 0x7f70c0118290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.455+0000 7f70c5562700 1 --2- 172.21.15.110:0/2828840836 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f70c00ab750 0x7f70c0117d50 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.455+0000 7f70c5562700 1 -- 172.21.15.110:0/2828840836 >> 172.21.15.110:0/2828840836 conn(0x7f70c0067670 msgr2=0x7f70c00aa820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:44.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.455+0000 7f3fc3fff700 1 -- 172.21.15.110:0/3196407668 shutdown_connections 2024-09-15T03:36:44.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.455+0000 7f3fc3fff700 1 -- 172.21.15.110:0/3196407668 wait complete. 2024-09-15T03:36:44.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.455+0000 7f70c5562700 1 -- 172.21.15.110:0/2828840836 shutdown_connections 2024-09-15T03:36:44.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.455+0000 7f70c5562700 1 -- 172.21.15.110:0/2828840836 wait complete. 2024-09-15T03:36:44.465 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.459+0000 7f1d10072700 1 -- 172.21.15.110:0/3895318291 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1d0806cfd0 con 0x7f1d080ad0f0 2024-09-15T03:36:44.465 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.459+0000 7f1cfdffb700 1 -- 172.21.15.110:0/3895318291 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1d0806cfd0 con 0x7f1d080ad0f0 2024-09-15T03:36:44.465 INFO:teuthology.orchestra.run.smithi110.stdout:55834574851 2024-09-15T03:36:44.467 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.1 2024-09-15T03:36:44.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.463+0000 7f1d10072700 1 -- 172.21.15.110:0/3895318291 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f1cec067b00 msgr2=0x7f1cec069fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:44.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.463+0000 7f1d10072700 1 --2- 172.21.15.110:0/3895318291 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f1cec067b00 0x7f1cec069fb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1d0400d440 tx=0x7f1d0400f040 comp rx=0 tx=0).stop 2024-09-15T03:36:44.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.463+0000 7f1d10072700 1 -- 172.21.15.110:0/3895318291 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d080ad0f0 msgr2=0x7f1d0806da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:44.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.463+0000 7f1d10072700 1 --2- 172.21.15.110:0/3895318291 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d080ad0f0 0x7f1d0806da50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1d0806f6c0 tx=0x7f1cf8017000 comp rx=0 tx=0).stop 2024-09-15T03:36:44.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.463+0000 7f57f6ac7700 1 -- 172.21.15.110:0/1289558709 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f57d0005630 con 0x7f57f0141720 2024-09-15T03:36:44.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.463+0000 7f1d10072700 1 -- 172.21.15.110:0/3895318291 shutdown_connections 2024-09-15T03:36:44.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.463+0000 7f1d10072700 1 --2- 172.21.15.110:0/3895318291 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f1cec067b00 0x7f1cec069fb0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.463+0000 7f1d10072700 1 --2- 172.21.15.110:0/3895318291 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f1d0806df90 0x7f1d0814c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.463+0000 7f1d10072700 1 --2- 172.21.15.110:0/3895318291 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d080ad0f0 0x7f1d0806da50 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.463+0000 7f57ecff9700 1 -- 172.21.15.110:0/1289558709 <== mon.2 v2:172.21.15.110:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f57e8055ae0 con 0x7f57f0141720 2024-09-15T03:36:44.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.463+0000 7f1d10072700 1 --2- 172.21.15.110:0/3895318291 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d080a9900 0x7f1d0813db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.463+0000 7f1d10072700 1 -- 172.21.15.110:0/3895318291 >> 172.21.15.110:0/3895318291 conn(0x7f1d08067670 msgr2=0x7f1d0813fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:44.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.463+0000 7f1d10072700 1 -- 172.21.15.110:0/3895318291 shutdown_connections 2024-09-15T03:36:44.471 INFO:teuthology.orchestra.run.smithi110.stdout:55834574851 2024-09-15T03:36:44.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.463+0000 7f1d10072700 1 -- 172.21.15.110:0/3895318291 wait complete. 2024-09-15T03:36:44.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.463+0000 7fdf55443700 1 -- 172.21.15.110:0/1758396172 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fdf30005630 con 0x7fdf5013da30 2024-09-15T03:36:44.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.463+0000 7fdf36ffd700 1 -- 172.21.15.110:0/1758396172 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fdf48015030 con 0x7fdf5013da30 2024-09-15T03:36:44.472 INFO:teuthology.orchestra.run.smithi110.stdout:60129542145 2024-09-15T03:36:44.472 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.467+0000 7fdf55443700 1 -- 172.21.15.110:0/1758396172 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fdf3c067a80 msgr2=0x7fdf3c069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:44.472 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.467+0000 7fdf55443700 1 --2- 172.21.15.110:0/1758396172 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fdf3c067a80 0x7fdf3c069f30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdf40007fd0 tx=0x7fdf400016f0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.473 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.467+0000 7fdf55443700 1 -- 172.21.15.110:0/1758396172 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf5013da30 msgr2=0x7fdf50151c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:44.473 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.467+0000 7fdf55443700 1 --2- 172.21.15.110:0/1758396172 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf5013da30 0x7fdf50151c90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdf4800c880 tx=0x7fdf48008040 comp rx=0 tx=0).stop 2024-09-15T03:36:44.473 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.467+0000 7fdf55443700 1 -- 172.21.15.110:0/1758396172 shutdown_connections 2024-09-15T03:36:44.473 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.467+0000 7fdf55443700 1 --2- 172.21.15.110:0/1758396172 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fdf3c067a80 0x7fdf3c069f30 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.473 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.467+0000 7fdf55443700 1 --2- 172.21.15.110:0/1758396172 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf5013da30 0x7fdf50151c90 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.473 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.467+0000 7fdf55443700 1 --2- 172.21.15.110:0/1758396172 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fdf50137140 0x7fdf50151750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.473 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.467+0000 7fdf55443700 1 --2- 172.21.15.110:0/1758396172 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdf500ad200 0x7fdf500af0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.473 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.467+0000 7fdf55443700 1 -- 172.21.15.110:0/1758396172 >> 172.21.15.110:0/1758396172 conn(0x7fdf50067a10 msgr2=0x7fdf5013eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:44.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.467+0000 7f57f6ac7700 1 -- 172.21.15.110:0/1289558709 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f57dc067ad0 msgr2=0x7f57dc069f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:44.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.467+0000 7f57f6ac7700 1 --2- 172.21.15.110:0/1289558709 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f57dc067ad0 0x7f57dc069f80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f57e0003080 tx=0x7f57e001f000 comp rx=0 tx=0).stop 2024-09-15T03:36:44.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.467+0000 7f57f6ac7700 1 -- 172.21.15.110:0/1289558709 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f57f0141720 msgr2=0x7f57f012c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:44.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.467+0000 7f57f6ac7700 1 --2- 172.21.15.110:0/1289558709 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f57f0141720 0x7f57f012c680 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f57e800cb40 tx=0x7f57e8001900 comp rx=0 tx=0).stop 2024-09-15T03:36:44.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.467+0000 7f57f6ac7700 1 -- 172.21.15.110:0/1289558709 shutdown_connections 2024-09-15T03:36:44.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.467+0000 7f57f6ac7700 1 --2- 172.21.15.110:0/1289558709 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f57dc067ad0 0x7f57dc069f80 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.467+0000 7f57f6ac7700 1 --2- 172.21.15.110:0/1289558709 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f57f0141720 0x7f57f012c680 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.467+0000 7f57f6ac7700 1 --2- 172.21.15.110:0/1289558709 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f57f01412f0 0x7f57f013dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.467+0000 7f57f6ac7700 1 --2- 172.21.15.110:0/1289558709 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f57f00ad280 0x7f57f0140410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:44.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.467+0000 7f57f6ac7700 1 -- 172.21.15.110:0/1289558709 >> 172.21.15.110:0/1289558709 conn(0x7f57f0067a10 msgr2=0x7f57f00af670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:44.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.467+0000 7f57f6ac7700 1 -- 172.21.15.110:0/1289558709 shutdown_connections 2024-09-15T03:36:44.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.467+0000 7f57f6ac7700 1 -- 172.21.15.110:0/1289558709 wait complete. 2024-09-15T03:36:44.476 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542146 for osd.7 2024-09-15T03:36:44.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.467+0000 7fdf55443700 1 -- 172.21.15.110:0/1758396172 shutdown_connections 2024-09-15T03:36:44.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:44.467+0000 7fdf55443700 1 -- 172.21.15.110:0/1758396172 wait complete. 2024-09-15T03:36:44.477 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542146 for osd.2 2024-09-15T03:36:44.481 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.0 2024-09-15T03:36:44.483 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.4 2024-09-15T03:36:44.484 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542147 got 60129542145 for osd.6 2024-09-15T03:36:45.459 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-15T03:36:45.459 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-15T03:36:45.468 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T03:36:45.477 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-15T03:36:45.479 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T03:36:45.481 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T03:36:45.484 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-15T03:36:45.484 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-15T03:36:45.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.587+0000 7f691bc5d700 1 Processor -- start 2024-09-15T03:36:45.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.587+0000 7f691bc5d700 1 -- start start 2024-09-15T03:36:45.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.591+0000 7f691bc5d700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f691413d120 0x7f691413d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:45.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.591+0000 7f691bc5d700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f6914139bc0 0x7f691413a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:45.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.591+0000 7f691bc5d700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f691406cb00 con 0x7f69140aced0 2024-09-15T03:36:45.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.591+0000 7f691bc5d700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f691406d860 con 0x7f691413d120 2024-09-15T03:36:45.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.591+0000 7f691bc5d700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69140ad2a0 con 0x7f6914139bc0 2024-09-15T03:36:45.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.591+0000 7f691a1fa700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f6914139bc0 0x7f691413a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:45.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.591+0000 7f691a1fa700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f6914139bc0 0x7f691413a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3301/0 says I am v2:172.21.15.110:38716/0 (socket says 172.21.15.110:38716) 2024-09-15T03:36:45.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.591+0000 7f691a1fa700 1 -- 172.21.15.110:0/1673032172 learned_addr learned my addr 172.21.15.110:0/1673032172 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:36:45.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.591+0000 7f691a1fa700 1 -- 172.21.15.110:0/1673032172 >> v2:172.21.15.177:3300/0 conn(0x7f691413d120 msgr2=0x7f691413d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:36:45.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.591+0000 7f691a1fa700 1 --2- 172.21.15.110:0/1673032172 >> v2:172.21.15.177:3300/0 conn(0x7f691413d120 0x7f691413d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.591+0000 7f691a1fa700 1 -- 172.21.15.110:0/1673032172 >> v1:172.21.15.177:6789/0 conn(0x7f69140aced0 legacy=0x7f691406c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:36:45.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.591+0000 7f691a1fa700 1 -- 172.21.15.110:0/1673032172 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f691406df60 con 0x7f6914139bc0 2024-09-15T03:36:45.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.591+0000 7f691a1fa700 1 --2- 172.21.15.110:0/1673032172 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f6914139bc0 0x7f691413a040 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f690c00d1a0 tx=0x7f690c007690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9219077e0a1405d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:45.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.591+0000 7f69189f7700 1 -- 172.21.15.110:0/1673032172 <== mon.2 v2:172.21.15.110:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f690c010070 con 0x7f6914139bc0 2024-09-15T03:36:45.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.591+0000 7f69189f7700 1 -- 172.21.15.110:0/1673032172 <== mon.2 v2:172.21.15.110:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f690c00dc70 con 0x7f6914139bc0 2024-09-15T03:36:45.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.591+0000 7f691bc5d700 1 -- 172.21.15.110:0/1673032172 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f6914139bc0 msgr2=0x7f691413a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:45.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.591+0000 7f691bc5d700 1 --2- 172.21.15.110:0/1673032172 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f6914139bc0 0x7f691413a040 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f690c00d1a0 tx=0x7f690c007690 comp rx=0 tx=0).stop 2024-09-15T03:36:45.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.591+0000 7f691bc5d700 1 -- 172.21.15.110:0/1673032172 shutdown_connections 2024-09-15T03:36:45.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.591+0000 7f691bc5d700 1 --2- 172.21.15.110:0/1673032172 >> v2:172.21.15.177:3300/0 conn(0x7f691413d120 0x7f691413d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.591+0000 7f691bc5d700 1 --2- 172.21.15.110:0/1673032172 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f6914139bc0 0x7f691413a040 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.591+0000 7f691bc5d700 1 -- 172.21.15.110:0/1673032172 >> 172.21.15.110:0/1673032172 conn(0x7f6914067670 msgr2=0x7f6914067a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:45.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.595+0000 7f691bc5d700 1 -- 172.21.15.110:0/1673032172 shutdown_connections 2024-09-15T03:36:45.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.595+0000 7f691bc5d700 1 -- 172.21.15.110:0/1673032172 wait complete. 2024-09-15T03:36:45.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.595+0000 7f691bc5d700 1 Processor -- start 2024-09-15T03:36:45.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.595+0000 7f691bc5d700 1 -- start start 2024-09-15T03:36:45.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.595+0000 7ff0c9765700 1 Processor -- start 2024-09-15T03:36:45.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.595+0000 7f691bc5d700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f69140aced0 0x7f69140a79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:45.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.595+0000 7f691bc5d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f691413d120 0x7f691414c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:45.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.595+0000 7f691bc5d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f691414cb20 0x7f69140a8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:45.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.595+0000 7f691bc5d700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69140a8090 con 0x7f691414cb20 2024-09-15T03:36:45.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.595+0000 7f691bc5d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69140a6e10 con 0x7f691413d120 2024-09-15T03:36:45.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.595+0000 7f691bc5d700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69140a5020 con 0x7f69140aced0 2024-09-15T03:36:45.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.595+0000 7f69199f9700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f69140aced0 0x7f69140a79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:45.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.595+0000 7f69199f9700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f69140aced0 0x7f69140a79e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3301/0 says I am v2:172.21.15.110:38728/0 (socket says 172.21.15.110:38728) 2024-09-15T03:36:45.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.595+0000 7f69199f9700 1 -- 172.21.15.110:0/2802668666 learned_addr learned my addr 172.21.15.110:0/2802668666 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:36:45.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.595+0000 7f691a1fa700 1 --2- 172.21.15.110:0/2802668666 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f691414cb20 0x7f69140a8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:45.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.595+0000 7ff0c9765700 1 -- start start 2024-09-15T03:36:45.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.595+0000 7ff0c9765700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ff0c40aced0 0x7ff0c406c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:45.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.595+0000 7ff0c9765700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7ff0c4139bc0 0x7ff0c413a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:45.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.595+0000 7ff0c9765700 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0c40ac650 con 0x7ff0c413d120 2024-09-15T03:36:45.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.595+0000 7ff0c9765700 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0c406cb00 con 0x7ff0c40aced0 2024-09-15T03:36:45.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.595+0000 7ff0c9765700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c406c7c0 con 0x7ff0c4139bc0 2024-09-15T03:36:45.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.595+0000 7ff0c27fc700 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7ff0c413d120 0x7ff0c413d550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:43744/0 (socket says 172.21.15.110:43744) 2024-09-15T03:36:45.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.595+0000 7ff0c27fc700 1 -- 172.21.15.110:0/1652234382 learned_addr learned my addr 172.21.15.110:0/1652234382 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-15T03:36:45.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.595+0000 7f69191f8700 1 --2- 172.21.15.110:0/2802668666 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f691413d120 0x7f691414c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:45.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.599+0000 7ff0c37fe700 1 --2- 172.21.15.110:0/1652234382 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7ff0c4139bc0 0x7ff0c413a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:45.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.599+0000 7f69199f9700 1 -- 172.21.15.110:0/2802668666 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f691413d120 msgr2=0x7f691414c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:45.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.599+0000 7f69199f9700 1 --2- 172.21.15.110:0/2802668666 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f691413d120 0x7f691414c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.599+0000 7f69199f9700 1 -- 172.21.15.110:0/2802668666 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f691414cb20 msgr2=0x7f69140a8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:45.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.599+0000 7f69199f9700 1 --2- 172.21.15.110:0/2802668666 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f691414cb20 0x7f69140a8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.599+0000 7f69199f9700 1 -- 172.21.15.110:0/2802668666 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f690c00be70 con 0x7f69140aced0 2024-09-15T03:36:45.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.599+0000 7ff0c37fe700 1 -- 172.21.15.110:0/1652234382 >> v2:172.21.15.110:3300/0 conn(0x7ff0c40aced0 msgr2=0x7ff0c406c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:36:45.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.599+0000 7ff0c2ffd700 1 --2- 172.21.15.110:0/1652234382 >> v2:172.21.15.110:3300/0 conn(0x7ff0c40aced0 0x7ff0c406c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:45.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.599+0000 7f69199f9700 1 --2- 172.21.15.110:0/2802668666 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f69140aced0 0x7f69140a79e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f691000b600 tx=0x7f69100091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:45.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.599+0000 7ff0c37fe700 1 --2- 172.21.15.110:0/1652234382 >> v2:172.21.15.110:3300/0 conn(0x7ff0c40aced0 0x7ff0c406c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.599+0000 7ff0c37fe700 1 -- 172.21.15.110:0/1652234382 >> v1:172.21.15.110:6789/0 conn(0x7ff0c413d120 legacy=0x7ff0c413d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:45.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.599+0000 7ff0c37fe700 1 -- 172.21.15.110:0/1652234382 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c406d860 con 0x7ff0c4139bc0 2024-09-15T03:36:45.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.599+0000 7f6909ffb700 1 -- 172.21.15.110:0/2802668666 <== mon.2 v2:172.21.15.110:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6910010840 con 0x7f69140aced0 2024-09-15T03:36:45.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.599+0000 7ff0c37fe700 1 --2- 172.21.15.110:0/1652234382 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7ff0c4139bc0 0x7ff0c413a040 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc009d50 tx=0x7ff0bc009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b156c839ab8e1037 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:45.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.599+0000 7ff0c2ffd700 1 --2- 172.21.15.110:0/1652234382 >> v2:172.21.15.110:3300/0 conn(0x7ff0c40aced0 0x7ff0c406c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T03:36:45.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.599+0000 7ff0c1ffb700 1 -- 172.21.15.110:0/1652234382 <== mon.2 v2:172.21.15.110:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc00cb70 con 0x7ff0c4139bc0 2024-09-15T03:36:45.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.599+0000 7ff0c1ffb700 1 -- 172.21.15.110:0/1652234382 <== mon.2 v2:172.21.15.110:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0bc00ccd0 con 0x7ff0c4139bc0 2024-09-15T03:36:45.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.599+0000 7ff0c1ffb700 1 -- 172.21.15.110:0/1652234382 <== mon.2 v2:172.21.15.110:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc011650 con 0x7ff0c4139bc0 2024-09-15T03:36:45.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.599+0000 7ff0c9765700 1 -- 172.21.15.110:0/1652234382 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7ff0c4139bc0 msgr2=0x7ff0c413a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:45.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.599+0000 7ff0c9765700 1 --2- 172.21.15.110:0/1652234382 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7ff0c4139bc0 0x7ff0c413a040 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc009d50 tx=0x7ff0bc009780 comp rx=0 tx=0).stop 2024-09-15T03:36:45.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.599+0000 7f691bc5d700 1 -- 172.21.15.110:0/2802668666 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69140a8a60 con 0x7f69140aced0 2024-09-15T03:36:45.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.599+0000 7f691bc5d700 1 -- 172.21.15.110:0/2802668666 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69141488e0 con 0x7f69140aced0 2024-09-15T03:36:45.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.599+0000 7f6909ffb700 1 -- 172.21.15.110:0/2802668666 <== mon.2 v2:172.21.15.110:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69100109c0 con 0x7f69140aced0 2024-09-15T03:36:45.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.599+0000 7f6909ffb700 1 -- 172.21.15.110:0/2802668666 <== mon.2 v2:172.21.15.110:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6910010d80 con 0x7f69140aced0 2024-09-15T03:36:45.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.599+0000 7ff0c9765700 1 -- 172.21.15.110:0/1652234382 shutdown_connections 2024-09-15T03:36:45.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.599+0000 7ff0c9765700 1 --2- 172.21.15.110:0/1652234382 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7ff0c4139bc0 0x7ff0c413a040 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.599+0000 7ff0c9765700 1 --2- 172.21.15.110:0/1652234382 >> v2:172.21.15.110:3300/0 conn(0x7ff0c40aced0 0x7ff0c406c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.599+0000 7ff0c9765700 1 -- 172.21.15.110:0/1652234382 >> 172.21.15.110:0/1652234382 conn(0x7ff0c4067670 msgr2=0x7ff0c4067a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:45.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.599+0000 7ff0c9765700 1 -- 172.21.15.110:0/1652234382 shutdown_connections 2024-09-15T03:36:45.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.603+0000 7f691bc5d700 1 -- 172.21.15.110:0/2802668666 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68f40050b0 con 0x7f69140aced0 2024-09-15T03:36:45.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.603+0000 7f6909ffb700 1 -- 172.21.15.110:0/2802668666 <== mon.2 v2:172.21.15.110:3301/0 4 ==== mgrmap(e 5) v1 ==== 85847+0+0 (secure 0 0 0) 0x7f6910016070 con 0x7f69140aced0 2024-09-15T03:36:45.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.603+0000 7f6909ffb700 1 --2- 172.21.15.110:0/2802668666 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f68f8067b80 0x7f68f806a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:45.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.603+0000 7f6909ffb700 1 -- 172.21.15.110:0/2802668666 <== mon.2 v2:172.21.15.110:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5262+0+0 (secure 0 0 0) 0x7f6910088b30 con 0x7f69140aced0 2024-09-15T03:36:45.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.603+0000 7ff0c9765700 1 -- 172.21.15.110:0/1652234382 wait complete. 2024-09-15T03:36:45.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.603+0000 7ff0c9765700 1 Processor -- start 2024-09-15T03:36:45.611 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.603+0000 7ff0c9765700 1 -- start start 2024-09-15T03:36:45.611 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.603+0000 7ff0c9765700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff0c40aced0 0x7ff0c4151410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:45.611 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.603+0000 7ff0c9765700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7ff0c4139bc0 0x7ff0c4137780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:45.611 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.603+0000 7ff0c9765700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff0c413d120 0x7ff0c4138370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:45.611 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.603+0000 7ff0c9765700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c40a4d70 con 0x7ff0c413d120 2024-09-15T03:36:45.611 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.603+0000 7ff0c9765700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c4137e50 con 0x7ff0c40aced0 2024-09-15T03:36:45.611 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.603+0000 7ff0c9765700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c4151ed0 con 0x7ff0c4139bc0 2024-09-15T03:36:45.611 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.603+0000 7ff0c2ffd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff0c40aced0 0x7ff0c4151410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:45.611 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.607+0000 7ff0c2ffd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff0c40aced0 0x7ff0c4151410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.110:33038/0 (socket says 172.21.15.110:33038) 2024-09-15T03:36:45.611 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.607+0000 7ff0c27fc700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7ff0c4139bc0 0x7ff0c4137780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:45.612 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.607+0000 7ff0c2ffd700 1 -- 172.21.15.110:0/517893745 learned_addr learned my addr 172.21.15.110:0/517893745 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:36:45.612 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.607+0000 7ff0c37fe700 1 --2- 172.21.15.110:0/517893745 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff0c413d120 0x7ff0c4138370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:45.612 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.607+0000 7ff0c2ffd700 1 -- 172.21.15.110:0/517893745 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7ff0c4139bc0 msgr2=0x7ff0c4137780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:45.612 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.607+0000 7ff0c2ffd700 1 --2- 172.21.15.110:0/517893745 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7ff0c4139bc0 0x7ff0c4137780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.612 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.607+0000 7ff0c2ffd700 1 -- 172.21.15.110:0/517893745 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff0c413d120 msgr2=0x7ff0c4138370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:45.612 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.607+0000 7ff0c2ffd700 1 --2- 172.21.15.110:0/517893745 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff0c413d120 0x7ff0c4138370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.612 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.607+0000 7ff0c2ffd700 1 -- 172.21.15.110:0/517893745 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0bc00d040 con 0x7ff0c40aced0 2024-09-15T03:36:45.612 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.607+0000 7f69191f8700 1 --2- 172.21.15.110:0/2802668666 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f68f8067b80 0x7f68f806a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:45.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.607+0000 7ff0c2ffd700 1 --2- 172.21.15.110:0/517893745 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff0c40aced0 0x7ff0c4151410 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff0b800c830 tx=0x7ff0b8008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:45.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.607+0000 7f69191f8700 1 --2- 172.21.15.110:0/2802668666 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f68f8067b80 0x7f68f806a030 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f69140a7290 tx=0x7f6904008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:45.614 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.607+0000 7ff0b2ffd700 1 -- 172.21.15.110:0/517893745 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b8011600 con 0x7ff0c40aced0 2024-09-15T03:36:45.614 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.607+0000 7ff0c9765700 1 -- 172.21.15.110:0/517893745 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0c4135270 con 0x7ff0c40aced0 2024-09-15T03:36:45.614 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.607+0000 7ff0b2ffd700 1 -- 172.21.15.110:0/517893745 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0b8011760 con 0x7ff0c40aced0 2024-09-15T03:36:45.614 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.607+0000 7ff0b2ffd700 1 -- 172.21.15.110:0/517893745 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b8011b20 con 0x7ff0c40aced0 2024-09-15T03:36:45.614 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.607+0000 7ff0c9765700 1 -- 172.21.15.110:0/517893745 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0c4138a50 con 0x7ff0c40aced0 2024-09-15T03:36:45.614 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.607+0000 7ff0c9765700 1 -- 172.21.15.110:0/517893745 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0a00050b0 con 0x7ff0c40aced0 2024-09-15T03:36:45.614 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.611+0000 7ff0b2ffd700 1 -- 172.21.15.110:0/517893745 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 5) v1 ==== 85847+0+0 (secure 0 0 0) 0x7ff0b8024480 con 0x7ff0c40aced0 2024-09-15T03:36:45.615 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.611+0000 7ff0b2ffd700 1 --2- 172.21.15.110:0/517893745 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7ff0a8067a80 0x7ff0a8069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:45.615 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.611+0000 7ff0b2ffd700 1 -- 172.21.15.110:0/517893745 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5262+0+0 (secure 0 0 0) 0x7ff0b8085910 con 0x7ff0c40aced0 2024-09-15T03:36:45.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.611+0000 7ff0c27fc700 1 --2- 172.21.15.110:0/517893745 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7ff0a8067a80 0x7ff0a8069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:45.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.611+0000 7f6909ffb700 1 -- 172.21.15.110:0/2802668666 <== mon.2 v2:172.21.15.110:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f6910056060 con 0x7f69140aced0 2024-09-15T03:36:45.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.611+0000 7ff0c27fc700 1 --2- 172.21.15.110:0/517893745 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7ff0a8067a80 0x7ff0a8069f30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff0c40a4ca0 tx=0x7ff0b4000bc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:45.624 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.619+0000 7ff0b2ffd700 1 -- 172.21.15.110:0/517893745 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7ff0b8052bd0 con 0x7ff0c40aced0 2024-09-15T03:36:45.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.639+0000 7f9570468700 1 Processor -- start 2024-09-15T03:36:45.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.639+0000 7f9570468700 1 -- start start 2024-09-15T03:36:45.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.639+0000 7f9570468700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f956806be40 0x7f956806e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:45.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.639+0000 7f9570468700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f9568145400 0x7f9568145880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:45.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.639+0000 7f9570468700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f956806cf30 con 0x7f95680acd80 2024-09-15T03:36:45.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.639+0000 7f9570468700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f956806d210 con 0x7f9568145400 2024-09-15T03:36:45.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.639+0000 7f9570468700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f956806d4f0 con 0x7f956806be40 2024-09-15T03:36:45.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.639+0000 7f956da03700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f956806be40 0x7f956806e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:45.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.639+0000 7f956ea05700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f9568145400 0x7f9568145880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:45.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.639+0000 7f956ea05700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f9568145400 0x7f9568145880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.110:33052/0 (socket says 172.21.15.110:33052) 2024-09-15T03:36:45.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.639+0000 7f956ea05700 1 -- 172.21.15.110:0/1194134019 learned_addr learned my addr 172.21.15.110:0/1194134019 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:36:45.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.639+0000 7f956ea05700 1 -- 172.21.15.110:0/1194134019 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f956806be40 msgr2=0x7f956806e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:36:45.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.639+0000 7f956ea05700 1 --2- 172.21.15.110:0/1194134019 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f956806be40 0x7f956806e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.639+0000 7f956ea05700 1 -- 172.21.15.110:0/1194134019 >> v1:172.21.15.177:6789/0 conn(0x7f95680acd80 legacy=0x7f956806dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:36:45.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.643+0000 7f956ea05700 1 -- 172.21.15.110:0/1194134019 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f956806c360 con 0x7f9568145400 2024-09-15T03:36:45.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.643+0000 7f956ea05700 1 --2- 172.21.15.110:0/1194134019 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9568145400 0x7f9568145880 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9560009f50 tx=0x7f9560009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4288340aee01b560 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:45.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.643+0000 7f956d202700 1 -- 172.21.15.110:0/1194134019 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956000e070 con 0x7f9568145400 2024-09-15T03:36:45.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.643+0000 7f956d202700 1 -- 172.21.15.110:0/1194134019 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f956000b980 con 0x7f9568145400 2024-09-15T03:36:45.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.643+0000 7f956d202700 1 -- 172.21.15.110:0/1194134019 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95600092e0 con 0x7f9568145400 2024-09-15T03:36:45.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.643+0000 7f9570468700 1 -- 172.21.15.110:0/1194134019 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9568145400 msgr2=0x7f9568145880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:45.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.643+0000 7f9570468700 1 --2- 172.21.15.110:0/1194134019 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9568145400 0x7f9568145880 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9560009f50 tx=0x7f9560009840 comp rx=0 tx=0).stop 2024-09-15T03:36:45.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.643+0000 7f9570468700 1 -- 172.21.15.110:0/1194134019 shutdown_connections 2024-09-15T03:36:45.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.643+0000 7f9570468700 1 --2- 172.21.15.110:0/1194134019 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9568145400 0x7f9568145880 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.643+0000 7f9570468700 1 --2- 172.21.15.110:0/1194134019 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f956806be40 0x7f956806e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.643+0000 7f9570468700 1 -- 172.21.15.110:0/1194134019 >> 172.21.15.110:0/1194134019 conn(0x7f9568067670 msgr2=0x7f9568067a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:45.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.643+0000 7fb7adeab700 1 Processor -- start 2024-09-15T03:36:45.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.643+0000 7f9570468700 1 -- 172.21.15.110:0/1194134019 shutdown_connections 2024-09-15T03:36:45.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.643+0000 7f9570468700 1 -- 172.21.15.110:0/1194134019 wait complete. 2024-09-15T03:36:45.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.643+0000 7fb7adeab700 1 -- start start 2024-09-15T03:36:45.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.643+0000 7fe4a816a700 1 Processor -- start 2024-09-15T03:36:45.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.643+0000 7fb7adeab700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fb7a80aced0 0x7fb7a806c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:45.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.643+0000 7fb7adeab700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fb7a8139bc0 0x7fb7a813a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:45.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.643+0000 7fb7adeab700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7a80ac650 con 0x7fb7a813d120 2024-09-15T03:36:45.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.643+0000 7fb7a7fff700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fb7a8139bc0 0x7fb7a813a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:45.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.643+0000 7fb7a7fff700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fb7a8139bc0 0x7fb7a813a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3301/0 says I am v2:172.21.15.110:38752/0 (socket says 172.21.15.110:38752) 2024-09-15T03:36:45.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.643+0000 7fb7a7fff700 1 -- 172.21.15.110:0/922667974 learned_addr learned my addr 172.21.15.110:0/922667974 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:36:45.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.643+0000 7fb7a77fe700 1 --2- 172.21.15.110:0/922667974 >> v2:172.21.15.110:3300/0 conn(0x7fb7a80aced0 0x7fb7a806c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:45.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.643+0000 7fb7adeab700 1 -- 172.21.15.110:0/922667974 --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7a806cb00 con 0x7fb7a80aced0 2024-09-15T03:36:45.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.643+0000 7fb7adeab700 1 -- 172.21.15.110:0/922667974 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a806c7c0 con 0x7fb7a8139bc0 2024-09-15T03:36:45.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.643+0000 7fb7a7fff700 1 -- 172.21.15.110:0/922667974 >> v2:172.21.15.110:3300/0 conn(0x7fb7a80aced0 msgr2=0x7fb7a806c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:36:45.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.643+0000 7fb7a7fff700 1 --2- 172.21.15.110:0/922667974 >> v2:172.21.15.110:3300/0 conn(0x7fb7a80aced0 0x7fb7a806c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.643+0000 7fb7a7fff700 1 -- 172.21.15.110:0/922667974 >> v1:172.21.15.177:6789/0 conn(0x7fb7a813d120 legacy=0x7fb7a813d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:36:45.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.643+0000 7fb7a7fff700 1 -- 172.21.15.110:0/922667974 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7a806d860 con 0x7fb7a8139bc0 2024-09-15T03:36:45.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.643+0000 7fe4a816a700 1 -- start start 2024-09-15T03:36:45.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.643+0000 7fe4a816a700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fe4a00aced0 0x7fe4a006c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:45.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.643+0000 7fe4a816a700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fe4a013d120 0x7fe4a013d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:45.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.643+0000 7fe4a816a700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fe4a00aa7a0 0x7fe4a00aac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:45.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.643+0000 7fe4a816a700 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4a006cb00 con 0x7fe4a00aced0 2024-09-15T03:36:45.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.643+0000 7fe4a816a700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4a006c7c0 con 0x7fe4a013d120 2024-09-15T03:36:45.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.643+0000 7fe4a816a700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a006d860 con 0x7fe4a00aa7a0 2024-09-15T03:36:45.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.643+0000 7f9570468700 1 Processor -- start 2024-09-15T03:36:45.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.643+0000 7f9570468700 1 -- start start 2024-09-15T03:36:45.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.643+0000 7fe4a5f06700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fe4a00aced0 0x7fe4a006c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:45.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.643+0000 7fe4a5f06700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fe4a00aced0 0x7fe4a006c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44230/0 (socket says 172.21.15.110:44230) 2024-09-15T03:36:45.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.643+0000 7fe4a5f06700 1 -- 172.21.15.110:0/440670643 learned_addr learned my addr 172.21.15.110:0/440670643 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:36:45.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7fe4a6707700 1 --2- 172.21.15.110:0/440670643 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fe4a00aa7a0 0x7fe4a00aac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:45.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7fb7a7fff700 1 --2- 172.21.15.110:0/922667974 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fb7a8139bc0 0x7fb7a813a040 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb7a0006360 tx=0x7fb7a0007e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f13ca1d7425bd861 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:45.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7f9570468700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f956806be40 0x7f9568146270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:45.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7f9570468700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95680acd80 0x7f95681336b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:45.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7f9570468700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9568145400 0x7f95680a72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:45.655 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7f9570468700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9568145ec0 con 0x7f95680acd80 2024-09-15T03:36:45.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7f9570468700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95680a58b0 con 0x7f9568145400 2024-09-15T03:36:45.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7f9570468700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f956814c250 con 0x7f956806be40 2024-09-15T03:36:45.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7fe4a6707700 1 -- 172.21.15.110:0/440670643 >> v2:172.21.15.177:3300/0 conn(0x7fe4a013d120 msgr2=0x7fe4a013d550 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:36:45.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7fe4a6707700 1 --2- 172.21.15.110:0/440670643 >> v2:172.21.15.177:3300/0 conn(0x7fe4a013d120 0x7fe4a013d550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7fe4a6707700 1 -- 172.21.15.110:0/440670643 >> v2:172.21.15.110:3300/0 conn(0x7fe4a00aced0 msgr2=0x7fe4a006c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:36:45.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7fe4a6707700 1 --2- 172.21.15.110:0/440670643 >> v2:172.21.15.110:3300/0 conn(0x7fe4a00aced0 0x7fe4a006c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7fe4a6707700 1 -- 172.21.15.110:0/440670643 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4a00ad2a0 con 0x7fe4a00aa7a0 2024-09-15T03:36:45.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7f956e204700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f956806be40 0x7f9568146270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:45.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7f956e204700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f956806be40 0x7f9568146270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3301/0 says I am v2:172.21.15.110:38774/0 (socket says 172.21.15.110:38774) 2024-09-15T03:36:45.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7f956e204700 1 -- 172.21.15.110:0/3418945193 learned_addr learned my addr 172.21.15.110:0/3418945193 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:36:45.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7f956da03700 1 --2- 172.21.15.110:0/3418945193 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95680acd80 0x7f95681336b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:45.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7f956ea05700 1 --2- 172.21.15.110:0/3418945193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9568145400 0x7f95680a72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:45.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7f956e204700 1 -- 172.21.15.110:0/3418945193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9568145400 msgr2=0x7f95680a72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:45.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7f956e204700 1 --2- 172.21.15.110:0/3418945193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9568145400 0x7f95680a72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7f956e204700 1 -- 172.21.15.110:0/3418945193 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95680acd80 msgr2=0x7f95681336b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:45.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7f956e204700 1 --2- 172.21.15.110:0/3418945193 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95680acd80 0x7f95681336b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7f956e204700 1 -- 172.21.15.110:0/3418945193 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9560009ae0 con 0x7f956806be40 2024-09-15T03:36:45.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7f956da03700 1 --2- 172.21.15.110:0/3418945193 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95680acd80 0x7f95681336b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T03:36:45.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7fe4a6707700 1 --2- 172.21.15.110:0/440670643 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fe4a00aa7a0 0x7fe4a00aac20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe4a00abf50 tx=0x7fe498009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b6a1b5b38ac18b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:45.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7f956e204700 1 --2- 172.21.15.110:0/3418945193 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f956806be40 0x7f9568146270 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f955800c5d0 tx=0x7f9558007f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:45.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7fe4a4f04700 1 -- 172.21.15.110:0/440670643 <== mon.2 v2:172.21.15.110:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe49800e070 con 0x7fe4a00aa7a0 2024-09-15T03:36:45.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7fe4a4f04700 1 -- 172.21.15.110:0/440670643 <== mon.2 v2:172.21.15.110:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe49800bb60 con 0x7fe4a00aa7a0 2024-09-15T03:36:45.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7fcf62286700 1 Processor -- start 2024-09-15T03:36:45.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7f01b7afc700 1 Processor -- start 2024-09-15T03:36:45.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7fe4a4f04700 1 -- 172.21.15.110:0/440670643 <== mon.2 v2:172.21.15.110:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe498012430 con 0x7fe4a00aa7a0 2024-09-15T03:36:45.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7fb7a67fc700 1 -- 172.21.15.110:0/922667974 <== mon.2 v2:172.21.15.110:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a000dcc0 con 0x7fb7a8139bc0 2024-09-15T03:36:45.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7fb7a67fc700 1 -- 172.21.15.110:0/922667974 <== mon.2 v2:172.21.15.110:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7a000de20 con 0x7fb7a8139bc0 2024-09-15T03:36:45.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7fe4a816a700 1 -- 172.21.15.110:0/440670643 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fe4a00aa7a0 msgr2=0x7fe4a00aac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:45.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7fe4a816a700 1 --2- 172.21.15.110:0/440670643 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fe4a00aa7a0 0x7fe4a00aac20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe4a00abf50 tx=0x7fe498009780 comp rx=0 tx=0).stop 2024-09-15T03:36:45.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7f955e7fc700 1 -- 172.21.15.110:0/3418945193 <== mon.2 v2:172.21.15.110:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9558011590 con 0x7f956806be40 2024-09-15T03:36:45.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7fe4a816a700 1 -- 172.21.15.110:0/440670643 shutdown_connections 2024-09-15T03:36:45.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7fe4a816a700 1 --2- 172.21.15.110:0/440670643 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fe4a00aa7a0 0x7fe4a00aac20 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7fe4a816a700 1 --2- 172.21.15.110:0/440670643 >> v2:172.21.15.177:3300/0 conn(0x7fe4a013d120 0x7fe4a013d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7fe4a816a700 1 --2- 172.21.15.110:0/440670643 >> v2:172.21.15.110:3300/0 conn(0x7fe4a00aced0 0x7fe4a006c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7fe4a816a700 1 -- 172.21.15.110:0/440670643 >> 172.21.15.110:0/440670643 conn(0x7fe4a0067670 msgr2=0x7fe4a0067a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:45.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7f955e7fc700 1 -- 172.21.15.110:0/3418945193 <== mon.2 v2:172.21.15.110:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9558011710 con 0x7f956806be40 2024-09-15T03:36:45.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7f01b7afc700 1 -- start start 2024-09-15T03:36:45.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7fb7a67fc700 1 -- 172.21.15.110:0/922667974 <== mon.2 v2:172.21.15.110:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a00072a0 con 0x7fb7a8139bc0 2024-09-15T03:36:45.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7fb7adeab700 1 -- 172.21.15.110:0/922667974 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fb7a8139bc0 msgr2=0x7fb7a813a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:45.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7fb7adeab700 1 --2- 172.21.15.110:0/922667974 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fb7a8139bc0 0x7fb7a813a040 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb7a0006360 tx=0x7fb7a0007e80 comp rx=0 tx=0).stop 2024-09-15T03:36:45.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7fb7adeab700 1 -- 172.21.15.110:0/922667974 shutdown_connections 2024-09-15T03:36:45.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7fb7adeab700 1 --2- 172.21.15.110:0/922667974 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fb7a8139bc0 0x7fb7a813a040 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7fb7adeab700 1 --2- 172.21.15.110:0/922667974 >> v2:172.21.15.110:3300/0 conn(0x7fb7a80aced0 0x7fb7a806c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7fb7adeab700 1 -- 172.21.15.110:0/922667974 >> 172.21.15.110:0/922667974 conn(0x7fb7a8067670 msgr2=0x7fb7a8067a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:45.670 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7f01b7afc700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f01b00acc40 0x7f01b006e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:45.670 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7f01b7afc700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f01b00ad010 0x7f01b006e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:45.670 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7f01b7afc700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f01b00ac3c0 0x7f01b00aa7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:45.670 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7f01b7afc700 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01b006c830 con 0x7f01b00ad010 2024-09-15T03:36:45.670 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7f01b7afc700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01b006d660 con 0x7f01b00acc40 2024-09-15T03:36:45.670 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7f01b7afc700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01b006ebc0 con 0x7f01b00ac3c0 2024-09-15T03:36:45.670 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7fcf62286700 1 -- start start 2024-09-15T03:36:45.670 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7fcf62286700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fcf5c0aced0 0x7fcf5c06c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:45.670 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7fcf62286700 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf5c0ac650 con 0x7fcf5c139bc0 2024-09-15T03:36:45.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7fcf62286700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf5c06cb00 con 0x7fcf5c13d120 2024-09-15T03:36:45.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7fcf62286700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf5c06c7c0 con 0x7fcf5c0aced0 2024-09-15T03:36:45.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7fb7adeab700 1 -- 172.21.15.110:0/922667974 shutdown_connections 2024-09-15T03:36:45.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7fcf60823700 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fcf5c139bc0 0x7fcf5c13a040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:43752/0 (socket says 172.21.15.110:43752) 2024-09-15T03:36:45.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7fcf60823700 1 -- 172.21.15.110:0/26495045 learned_addr learned my addr 172.21.15.110:0/26495045 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-15T03:36:45.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7f49cbdea700 1 Processor -- start 2024-09-15T03:36:45.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7fb7adeab700 1 -- 172.21.15.110:0/922667974 wait complete. 2024-09-15T03:36:45.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7f01b5097700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f01b00ad010 0x7f01b006e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:45.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7f49cbdea700 1 -- start start 2024-09-15T03:36:45.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7f01b5898700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f01b00acc40 0x7f01b006e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:45.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7f01b5898700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f01b00acc40 0x7f01b006e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.110:33080/0 (socket says 172.21.15.110:33080) 2024-09-15T03:36:45.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7f01b5898700 1 -- 172.21.15.110:0/4247106296 learned_addr learned my addr 172.21.15.110:0/4247106296 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:36:45.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7f955e7fc700 1 -- 172.21.15.110:0/3418945193 <== mon.2 v2:172.21.15.110:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9558011ad0 con 0x7f956806be40 2024-09-15T03:36:45.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7f9570468700 1 -- 172.21.15.110:0/3418945193 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9568133bf0 con 0x7f956806be40 2024-09-15T03:36:45.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7f9570468700 1 -- 172.21.15.110:0/3418945193 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95680a6500 con 0x7f956806be40 2024-09-15T03:36:45.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7f49cbdea700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f49c4068680 0x7f49c40ab260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:45.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7f49cbdea700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f49c4068a50 0x7f49c4138680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:45.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7f49cbdea700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f49c4138bc0 0x7f49c413ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:45.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7f49cbdea700 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49c4139060 con 0x7f49c4068a50 2024-09-15T03:36:45.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.651+0000 7f49cbdea700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49c40acbc0 con 0x7f49c4138bc0 2024-09-15T03:36:45.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.651+0000 7f49cbdea700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49c40ab9c0 con 0x7f49c4068680 2024-09-15T03:36:45.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.651+0000 7fb7adeab700 1 Processor -- start 2024-09-15T03:36:45.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.651+0000 7fb7adeab700 1 -- start start 2024-09-15T03:36:45.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7f01b5097700 1 -- 172.21.15.110:0/4247106296 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f01b00ac3c0 msgr2=0x7f01b00aa7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:36:45.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7fcf5affd700 1 -- 172.21.15.110:0/26495045 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 904672817 0 0) 0x7fcf5c06cb00 con 0x7fcf5c13d120 2024-09-15T03:36:45.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7fcf5affd700 1 -- 172.21.15.110:0/26495045 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf48003570 con 0x7fcf5c13d120 2024-09-15T03:36:45.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7fcf5affd700 1 -- 172.21.15.110:0/26495045 <== mon.1 v1:172.21.15.177:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 866426931 0 0) 0x7fcf48003570 con 0x7fcf5c13d120 2024-09-15T03:36:45.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7fcf5affd700 1 -- 172.21.15.110:0/26495045 >> v2:172.21.15.177:3300/0 conn(0x7fcf5c0aced0 msgr2=0x7fcf5c06c280 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T03:36:45.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7fcf5affd700 1 --2- 172.21.15.110:0/26495045 >> v2:172.21.15.177:3300/0 conn(0x7fcf5c0aced0 0x7fcf5c06c280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7fcf5affd700 1 -- 172.21.15.110:0/26495045 >> v1:172.21.15.110:6789/0 conn(0x7fcf5c139bc0 legacy=0x7fcf5c13a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:36:45.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7fcf5affd700 1 -- 172.21.15.110:0/26495045 --> v1:172.21.15.177:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf5c06d860 con 0x7fcf5c13d120 2024-09-15T03:36:45.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7fcf5affd700 1 -- 172.21.15.110:0/26495045 <== mon.1 v1:172.21.15.177:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 644009436 0 0) 0x7fcf44001470 con 0x7fcf5c13d120 2024-09-15T03:36:45.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7fcf5affd700 1 -- 172.21.15.110:0/26495045 >> v1:172.21.15.177:6789/0 conn(0x7fcf5c13d120 legacy=0x7fcf5c13d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:45.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7fcf5affd700 1 --2- 172.21.15.110:0/26495045 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf48003ae0 0x7fcf48003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:45.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7fcf5affd700 1 --2- 172.21.15.110:0/26495045 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fcf48004810 0x7fcf48004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:45.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7fcf5affd700 1 --2- 172.21.15.110:0/26495045 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf48009a50 0x7fcf4800be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:45.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7fcf5affd700 1 -- 172.21.15.110:0/26495045 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5c06c7c0 con 0x7fcf48003ae0 2024-09-15T03:36:45.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7fcf5affd700 1 -- 172.21.15.110:0/26495045 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf4800c3f0 con 0x7fcf48009a50 2024-09-15T03:36:45.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.647+0000 7fcf5affd700 1 -- 172.21.15.110:0/26495045 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf4800c5b0 con 0x7fcf48004810 2024-09-15T03:36:45.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.651+0000 7fcf5b7fe700 1 --2- 172.21.15.110:0/26495045 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fcf48004810 0x7fcf48004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:45.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.651+0000 7fcf60823700 1 --2- 172.21.15.110:0/26495045 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf48009a50 0x7fcf4800be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:45.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.651+0000 7fe4a816a700 1 -- 172.21.15.110:0/440670643 shutdown_connections 2024-09-15T03:36:45.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.651+0000 7fcf5b7fe700 1 -- 172.21.15.110:0/26495045 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf48009a50 msgr2=0x7fcf4800be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:36:45.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.651+0000 7fcf5b7fe700 1 --2- 172.21.15.110:0/26495045 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf48009a50 0x7fcf4800be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.651+0000 7fcf5b7fe700 1 -- 172.21.15.110:0/26495045 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf48003ae0 msgr2=0x7fcf48003ed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:36:45.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.651+0000 7f9570468700 1 -- 172.21.15.110:0/3418945193 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95680a6fe0 con 0x7f956806be40 2024-09-15T03:36:45.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.651+0000 7f01b6099700 1 --2- 172.21.15.110:0/4247106296 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f01b00ac3c0 0x7f01b00aa7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:45.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.651+0000 7f49ca387700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f49c4138bc0 0x7f49c413ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:45.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.651+0000 7f49ca387700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f49c4138bc0 0x7f49c413ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.110:33108/0 (socket says 172.21.15.110:33108) 2024-09-15T03:36:45.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.651+0000 7f49ca387700 1 -- 172.21.15.110:0/3421097773 learned_addr learned my addr 172.21.15.110:0/3421097773 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:36:45.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.651+0000 7f49c9385700 1 --2- 172.21.15.110:0/3421097773 >> v2:172.21.15.110:3300/0 conn(0x7f49c4068a50 0x7f49c4138680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:45.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.651+0000 7f49ca387700 1 -- 172.21.15.110:0/3421097773 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f49c4068680 msgr2=0x7f49c40ab260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:36:45.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.651+0000 7f49ca387700 1 --2- 172.21.15.110:0/3421097773 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f49c4068680 0x7f49c40ab260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.651+0000 7f49ca387700 1 -- 172.21.15.110:0/3421097773 >> v2:172.21.15.110:3300/0 conn(0x7f49c4068a50 msgr2=0x7f49c4138680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:36:45.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.651+0000 7f49ca387700 1 --2- 172.21.15.110:0/3421097773 >> v2:172.21.15.110:3300/0 conn(0x7f49c4068a50 0x7f49c4138680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.651+0000 7f49ca387700 1 -- 172.21.15.110:0/3421097773 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c40ace30 con 0x7f49c4138bc0 2024-09-15T03:36:45.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.651+0000 7f49c9385700 1 --2- 172.21.15.110:0/3421097773 >> v2:172.21.15.110:3300/0 conn(0x7f49c4068a50 0x7f49c4138680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T03:36:45.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.651+0000 7fe4a816a700 1 -- 172.21.15.110:0/440670643 wait complete. 2024-09-15T03:36:45.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.651+0000 7fcf5bfff700 1 --2- 172.21.15.110:0/26495045 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf48003ae0 0x7fcf48003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:45.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.651+0000 7fcf5b7fe700 1 --2- 172.21.15.110:0/26495045 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf48003ae0 0x7fcf48003ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.651+0000 7fcf5b7fe700 1 -- 172.21.15.110:0/26495045 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf5c06d860 con 0x7fcf48004810 2024-09-15T03:36:45.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.651+0000 7fb7adeab700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7a80aced0 0x7fb7a8137660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:45.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.651+0000 7fb7adeab700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7a8139bc0 0x7fb7a80ae730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:45.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.651+0000 7fb7adeab700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fb7a813d120 0x7fb7a80a4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:45.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.651+0000 7f01b5097700 1 --2- 172.21.15.110:0/4247106296 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f01b00ac3c0 0x7f01b00aa7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.651+0000 7f955e7fc700 1 -- 172.21.15.110:0/3418945193 <== mon.2 v2:172.21.15.110:3301/0 4 ==== mgrmap(e 5) v1 ==== 85847+0+0 (secure 0 0 0) 0x7f955801c460 con 0x7f956806be40 2024-09-15T03:36:45.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.651+0000 7fb7adeab700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a80af660 con 0x7fb7a8139bc0 2024-09-15T03:36:45.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.651+0000 7fb7adeab700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a80ae2a0 con 0x7fb7a80aced0 2024-09-15T03:36:45.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.651+0000 7fb7adeab700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a80af8d0 con 0x7fb7a813d120 2024-09-15T03:36:45.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.651+0000 7fb7a6ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7a8139bc0 0x7fb7a80ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:45.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.651+0000 7fb7a6ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7a8139bc0 0x7fb7a80ae730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44276/0 (socket says 172.21.15.110:44276) 2024-09-15T03:36:45.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.651+0000 7fb7a6ffd700 1 -- 172.21.15.110:0/418402908 learned_addr learned my addr 172.21.15.110:0/418402908 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:36:45.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.651+0000 7f01b5097700 1 -- 172.21.15.110:0/4247106296 >> v2:172.21.15.177:3300/0 conn(0x7f01b00acc40 msgr2=0x7f01b006e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:36:45.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.651+0000 7fb7a7fff700 1 --2- 172.21.15.110:0/418402908 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fb7a813d120 0x7fb7a80a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:45.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.651+0000 7fb7a77fe700 1 --2- 172.21.15.110:0/418402908 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7a80aced0 0x7fb7a8137660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:45.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.651+0000 7fb7a6ffd700 1 -- 172.21.15.110:0/418402908 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fb7a813d120 msgr2=0x7fb7a80a4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:45.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.651+0000 7fb7a6ffd700 1 --2- 172.21.15.110:0/418402908 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fb7a813d120 0x7fb7a80a4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.651+0000 7fb7a6ffd700 1 -- 172.21.15.110:0/418402908 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7a80aced0 msgr2=0x7fb7a8137660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:45.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.651+0000 7fb7a6ffd700 1 --2- 172.21.15.110:0/418402908 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7a80aced0 0x7fb7a8137660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.684 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.651+0000 7fb7a6ffd700 1 -- 172.21.15.110:0/418402908 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7a0005ea0 con 0x7fb7a8139bc0 2024-09-15T03:36:45.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.651+0000 7f01b5097700 1 --2- 172.21.15.110:0/4247106296 >> v2:172.21.15.177:3300/0 conn(0x7f01b00acc40 0x7f01b006e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.651+0000 7f01b5097700 1 -- 172.21.15.110:0/4247106296 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01b006ed30 con 0x7f01b00ad010 2024-09-15T03:36:45.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.651+0000 7fb7a6ffd700 1 --2- 172.21.15.110:0/418402908 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7a8139bc0 0x7fb7a80ae730 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb79c00b610 tx=0x7fb79c0090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:45.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.651+0000 7f955e7fc700 1 --2- 172.21.15.110:0/3418945193 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f954c067b50 0x7f954c06a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:45.686 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.651+0000 7f01b6099700 1 --2- 172.21.15.110:0/4247106296 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f01b00ac3c0 0x7f01b00aa7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T03:36:45.686 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.651+0000 7f01b5097700 1 --2- 172.21.15.110:0/4247106296 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f01b00ad010 0x7f01b006e680 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f01a00098e0 tx=0x7f01a00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7e5b7667543e513 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:45.686 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.651+0000 7f956da03700 1 --2- 172.21.15.110:0/3418945193 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f954c067b50 0x7f954c06a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:45.686 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.651+0000 7f955e7fc700 1 -- 172.21.15.110:0/3418945193 <== mon.2 v2:172.21.15.110:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5262+0+0 (secure 0 0 0) 0x7f95580851d0 con 0x7f956806be40 2024-09-15T03:36:45.686 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.651+0000 7f49ca387700 1 --2- 172.21.15.110:0/3421097773 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49c4138bc0 0x7f49c413ea90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f49c40ac4e0 tx=0x7f49c0009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f7be0ce7ddb1653 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:45.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.651+0000 7fb78f7fe700 1 -- 172.21.15.110:0/418402908 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79c00f510 con 0x7fb7a8139bc0 2024-09-15T03:36:45.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.651+0000 7fb78f7fe700 1 -- 172.21.15.110:0/418402908 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb79c00f670 con 0x7fb7a8139bc0 2024-09-15T03:36:45.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.651+0000 7fb7adeab700 1 -- 172.21.15.110:0/418402908 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7a80addf0 con 0x7fb7a8139bc0 2024-09-15T03:36:45.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.651+0000 7fb78f7fe700 1 -- 172.21.15.110:0/418402908 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79c00fa30 con 0x7fb7a8139bc0 2024-09-15T03:36:45.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.651+0000 7fb7adeab700 1 -- 172.21.15.110:0/418402908 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7a80a4f70 con 0x7fb7a8139bc0 2024-09-15T03:36:45.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.651+0000 7f01b5898700 1 --2- 172.21.15.110:0/4247106296 >> v2:172.21.15.177:3300/0 conn(0x7f01b00acc40 0x7f01b006e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T03:36:45.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.651+0000 7f956da03700 1 --2- 172.21.15.110:0/3418945193 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f954c067b50 0x7f954c06a000 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f95680a5550 tx=0x7f95640091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:45.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.651+0000 7f01b4896700 1 -- 172.21.15.110:0/4247106296 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a0011630 con 0x7f01b00ad010 2024-09-15T03:36:45.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.655+0000 7f01b4896700 1 -- 172.21.15.110:0/4247106296 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01a0011790 con 0x7f01b00ad010 2024-09-15T03:36:45.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.655+0000 7f01b4896700 1 -- 172.21.15.110:0/4247106296 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a0011b30 con 0x7f01b00ad010 2024-09-15T03:36:45.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.655+0000 7fb7adeab700 1 -- 172.21.15.110:0/418402908 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7a8064bd0 con 0x7fb7a8139bc0 2024-09-15T03:36:45.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.655+0000 7f01b7afc700 1 -- 172.21.15.110:0/4247106296 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f01b00ad010 msgr2=0x7f01b006e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:45.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.655+0000 7f01b7afc700 1 --2- 172.21.15.110:0/4247106296 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f01b00ad010 0x7f01b006e680 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f01a00098e0 tx=0x7f01a00091a0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.655+0000 7fb78f7fe700 1 -- 172.21.15.110:0/418402908 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) v1 ==== 85847+0+0 (secure 0 0 0) 0x7fb79c024430 con 0x7fb7a8139bc0 2024-09-15T03:36:45.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.655+0000 7fb78f7fe700 1 --2- 172.21.15.110:0/418402908 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fb794067b50 0x7fb79406a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:45.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.655+0000 7fb78f7fe700 1 -- 172.21.15.110:0/418402908 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5262+0+0 (secure 0 0 0) 0x7fb79c085a10 con 0x7fb7a8139bc0 2024-09-15T03:36:45.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.655+0000 7fb7a77fe700 1 --2- 172.21.15.110:0/418402908 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fb794067b50 0x7fb79406a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:45.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.655+0000 7fb7a77fe700 1 --2- 172.21.15.110:0/418402908 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fb794067b50 0x7fb79406a000 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb798009d40 tx=0x7fb798008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:45.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.651+0000 7fe4a816a700 1 Processor -- start 2024-09-15T03:36:45.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.651+0000 7fcf60823700 1 -- 172.21.15.110:0/26495045 reap_dead start 2024-09-15T03:36:45.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.651+0000 7fcf5b7fe700 1 --2- 172.21.15.110:0/26495045 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fcf48004810 0x7fcf48004c80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fcf4400e600 tx=0x7fcf44002e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=974be25990c5a11b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:45.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.651+0000 7fcf5bfff700 1 --2- 172.21.15.110:0/26495045 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf48003ae0 0x7fcf48003ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T03:36:45.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.659+0000 7fe4a816a700 1 -- start start 2024-09-15T03:36:45.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.659+0000 7f01b7afc700 1 -- 172.21.15.110:0/4247106296 shutdown_connections 2024-09-15T03:36:45.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.659+0000 7f01b7afc700 1 --2- 172.21.15.110:0/4247106296 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f01b00ac3c0 0x7f01b00aa7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.659+0000 7f01b7afc700 1 --2- 172.21.15.110:0/4247106296 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f01b00ad010 0x7f01b006e680 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.659+0000 7f01b7afc700 1 --2- 172.21.15.110:0/4247106296 >> v2:172.21.15.177:3300/0 conn(0x7f01b00acc40 0x7f01b006e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.659+0000 7f01b7afc700 1 -- 172.21.15.110:0/4247106296 >> 172.21.15.110:0/4247106296 conn(0x7f01b0067810 msgr2=0x7f01b0067c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:45.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.659+0000 7fb78f7fe700 1 -- 172.21.15.110:0/418402908 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fb79c052e30 con 0x7fb7a8139bc0 2024-09-15T03:36:45.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.659+0000 7fe4a816a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe4a00aa7a0 0x7fe4a00aed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:45.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.659+0000 7fe4a816a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4a00aced0 0x7fe4a01517d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:45.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.659+0000 7f49c8b84700 1 -- 172.21.15.110:0/3421097773 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c000f360 con 0x7f49c4138bc0 2024-09-15T03:36:45.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.659+0000 7f49c8b84700 1 -- 172.21.15.110:0/3421097773 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49c0013070 con 0x7f49c4138bc0 2024-09-15T03:36:45.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.659+0000 7f49c8b84700 1 -- 172.21.15.110:0/3421097773 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c0009940 con 0x7f49c4138bc0 2024-09-15T03:36:45.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.659+0000 7f49cbdea700 1 -- 172.21.15.110:0/3421097773 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49c4138bc0 msgr2=0x7f49c413ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:45.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.659+0000 7f49cbdea700 1 --2- 172.21.15.110:0/3421097773 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49c4138bc0 0x7f49c413ea90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f49c40ac4e0 tx=0x7f49c0009250 comp rx=0 tx=0).stop 2024-09-15T03:36:45.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.659+0000 7f49cbdea700 1 -- 172.21.15.110:0/3421097773 shutdown_connections 2024-09-15T03:36:45.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.659+0000 7f49cbdea700 1 --2- 172.21.15.110:0/3421097773 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49c4138bc0 0x7f49c413ea90 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.659+0000 7f49cbdea700 1 --2- 172.21.15.110:0/3421097773 >> v2:172.21.15.110:3300/0 conn(0x7f49c4068a50 0x7f49c4138680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.659+0000 7f49cbdea700 1 --2- 172.21.15.110:0/3421097773 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f49c4068680 0x7f49c40ab260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.659+0000 7f49cbdea700 1 -- 172.21.15.110:0/3421097773 >> 172.21.15.110:0/3421097773 conn(0x7f49c40ad150 msgr2=0x7f49c40ad550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:45.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.659+0000 7fe4a816a700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fe4a013d120 0x7fe4a0154ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:45.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.659+0000 7fe4a816a700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a00a44d0 con 0x7fe4a00aced0 2024-09-15T03:36:45.695 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.659+0000 7fe4a816a700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a013c680 con 0x7fe4a00aa7a0 2024-09-15T03:36:45.695 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.659+0000 7fe4a816a700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a00a59a0 con 0x7fe4a013d120 2024-09-15T03:36:45.695 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.659+0000 7fe4a5f06700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe4a00aa7a0 0x7fe4a00aed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:45.695 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.659+0000 7fe4a6707700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fe4a013d120 0x7fe4a0154ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:45.695 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.659+0000 7fe4a5f06700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe4a00aa7a0 0x7fe4a00aed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.110:33126/0 (socket says 172.21.15.110:33126) 2024-09-15T03:36:45.695 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.659+0000 7fe4a5f06700 1 -- 172.21.15.110:0/3023401115 learned_addr learned my addr 172.21.15.110:0/3023401115 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:36:45.695 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.659+0000 7fe4a5705700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4a00aced0 0x7fe4a01517d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:45.695 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.659+0000 7f01b7afc700 1 -- 172.21.15.110:0/4247106296 shutdown_connections 2024-09-15T03:36:45.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.659+0000 7fe4a6707700 1 -- 172.21.15.110:0/3023401115 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe4a00aa7a0 msgr2=0x7fe4a00aed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:45.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.659+0000 7fe4a6707700 1 --2- 172.21.15.110:0/3023401115 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe4a00aa7a0 0x7fe4a00aed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.659+0000 7fe4a6707700 1 -- 172.21.15.110:0/3023401115 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4a00aced0 msgr2=0x7fe4a01517d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:45.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.659+0000 7fe4a6707700 1 --2- 172.21.15.110:0/3023401115 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4a00aced0 0x7fe4a01517d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.659+0000 7fe4a6707700 1 -- 172.21.15.110:0/3023401115 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe498009a20 con 0x7fe4a013d120 2024-09-15T03:36:45.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.659+0000 7fe4a5705700 1 --2- 172.21.15.110:0/3023401115 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4a00aced0 0x7fe4a01517d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T03:36:45.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.659+0000 7f01b7afc700 1 -- 172.21.15.110:0/4247106296 wait complete. 2024-09-15T03:36:45.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.659+0000 7fe4a6707700 1 --2- 172.21.15.110:0/3023401115 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fe4a013d120 0x7fe4a0154ce0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe4a0069660 tx=0x7fe498019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:45.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.659+0000 7f955e7fc700 1 -- 172.21.15.110:0/3418945193 <== mon.2 v2:172.21.15.110:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f95680a6fe0 con 0x7f956806be40 2024-09-15T03:36:45.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.659+0000 7f01b7afc700 1 Processor -- start 2024-09-15T03:36:45.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.659+0000 7f01b7afc700 1 -- start start 2024-09-15T03:36:45.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.663+0000 7f49cbdea700 1 -- 172.21.15.110:0/3421097773 shutdown_connections 2024-09-15T03:36:45.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.663+0000 7fe495ffb700 1 -- 172.21.15.110:0/3023401115 <== mon.2 v2:172.21.15.110:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe498012430 con 0x7fe4a013d120 2024-09-15T03:36:45.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.663+0000 7fe495ffb700 1 -- 172.21.15.110:0/3023401115 <== mon.2 v2:172.21.15.110:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe498021070 con 0x7fe4a013d120 2024-09-15T03:36:45.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.663+0000 7fe495ffb700 1 -- 172.21.15.110:0/3023401115 <== mon.2 v2:172.21.15.110:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe498026430 con 0x7fe4a013d120 2024-09-15T03:36:45.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.663+0000 7f49cbdea700 1 -- 172.21.15.110:0/3421097773 wait complete. 2024-09-15T03:36:45.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.663+0000 7f01b7afc700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f01b00ac3c0 0x7f01b00a7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:45.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.663+0000 7f01b7afc700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01b00acc40 0x7f01b0120680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:45.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.663+0000 7f01b7afc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f01b00ad010 0x7f01b0121930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:45.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.663+0000 7f01b5898700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f01b00ac3c0 0x7f01b00a7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:45.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.663+0000 7f01b7afc700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b0123a60 con 0x7f01b00ad010 2024-09-15T03:36:45.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.663+0000 7f01b5898700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f01b00ac3c0 0x7f01b00a7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3301/0 says I am v2:172.21.15.110:38840/0 (socket says 172.21.15.110:38840) 2024-09-15T03:36:45.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.663+0000 7f01b5898700 1 -- 172.21.15.110:0/1924997697 learned_addr learned my addr 172.21.15.110:0/1924997697 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:36:45.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.663+0000 7f01b5097700 1 --2- 172.21.15.110:0/1924997697 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01b00acc40 0x7f01b0120680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:45.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.663+0000 7f01b6099700 1 --2- 172.21.15.110:0/1924997697 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f01b00ad010 0x7f01b0121930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:45.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.663+0000 7f01b7afc700 1 -- 172.21.15.110:0/1924997697 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b01211f0 con 0x7f01b00acc40 2024-09-15T03:36:45.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.663+0000 7f01b7afc700 1 -- 172.21.15.110:0/1924997697 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01b0120bc0 con 0x7f01b00ac3c0 2024-09-15T03:36:45.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.663+0000 7f49cbdea700 1 Processor -- start 2024-09-15T03:36:45.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.663+0000 7fe4a816a700 1 -- 172.21.15.110:0/3023401115 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4a00a5ce0 con 0x7fe4a013d120 2024-09-15T03:36:45.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.663+0000 7fe4a816a700 1 -- 172.21.15.110:0/3023401115 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4a00a4920 con 0x7fe4a013d120 2024-09-15T03:36:45.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.663+0000 7f49cbdea700 1 -- start start 2024-09-15T03:36:45.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.663+0000 7f01b6099700 1 -- 172.21.15.110:0/1924997697 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f01b00ac3c0 msgr2=0x7f01b00a7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:45.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.663+0000 7f01b6099700 1 --2- 172.21.15.110:0/1924997697 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f01b00ac3c0 0x7f01b00a7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.663+0000 7f01b6099700 1 -- 172.21.15.110:0/1924997697 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01b00acc40 msgr2=0x7f01b0120680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:45.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.663+0000 7f01b6099700 1 --2- 172.21.15.110:0/1924997697 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01b00acc40 0x7f01b0120680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.663+0000 7f01b6099700 1 -- 172.21.15.110:0/1924997697 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01a40094f0 con 0x7f01b00ad010 2024-09-15T03:36:45.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.663+0000 7f01b6099700 1 --2- 172.21.15.110:0/1924997697 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f01b00ad010 0x7f01b0121930 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f01a800cf30 tx=0x7f01a8007840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:45.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.663+0000 7f49cbdea700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49c4068680 0x7f49c413b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:45.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.663+0000 7f49cbdea700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49c4068a50 0x7f49c406c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:45.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.663+0000 7f49cbdea700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f49c4138bc0 0x7f49c4134410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:45.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.663+0000 7f49cbdea700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c40a9e20 con 0x7f49c4068680 2024-09-15T03:36:45.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.663+0000 7f49cbdea700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c413c280 con 0x7f49c4068a50 2024-09-15T03:36:45.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.663+0000 7f49cbdea700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49c406d030 con 0x7f49c4138bc0 2024-09-15T03:36:45.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.663+0000 7f49c9b86700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49c4068680 0x7f49c413b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:45.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.663+0000 7f49ca387700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f49c4138bc0 0x7f49c4134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:45.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.663+0000 7f49c9b86700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49c4068680 0x7f49c413b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44306/0 (socket says 172.21.15.110:44306) 2024-09-15T03:36:45.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.663+0000 7f49c9b86700 1 -- 172.21.15.110:0/3782920615 learned_addr learned my addr 172.21.15.110:0/3782920615 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:36:45.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.663+0000 7f49ca387700 1 --2- 172.21.15.110:0/3782920615 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f49c4138bc0 0x7f49c4134410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3301/0 says I am v2:172.21.15.110:38846/0 (socket says 172.21.15.110:38846) 2024-09-15T03:36:45.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.663+0000 7f49c9385700 1 --2- 172.21.15.110:0/3782920615 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49c4068a50 0x7f49c406c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:45.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.663+0000 7f49c9b86700 1 -- 172.21.15.110:0/3782920615 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f49c4138bc0 msgr2=0x7f49c4134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:45.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.663+0000 7f49c9b86700 1 --2- 172.21.15.110:0/3782920615 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f49c4138bc0 0x7f49c4134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.663+0000 7f49c9b86700 1 -- 172.21.15.110:0/3782920615 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49c4068a50 msgr2=0x7f49c406c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:45.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.663+0000 7f49c9b86700 1 --2- 172.21.15.110:0/3782920615 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49c4068a50 0x7f49c406c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.663+0000 7f49c9b86700 1 -- 172.21.15.110:0/3782920615 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49bc009a20 con 0x7f49c4068680 2024-09-15T03:36:45.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.663+0000 7f49c9b86700 1 --2- 172.21.15.110:0/3782920615 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49c4068680 0x7f49c413b3c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f49b400b810 tx=0x7f49b40091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:45.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.663+0000 7f49b9ffb700 1 -- 172.21.15.110:0/3782920615 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b400f510 con 0x7f49c4068680 2024-09-15T03:36:45.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.663+0000 7f49b9ffb700 1 -- 172.21.15.110:0/3782920615 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b400f670 con 0x7f49c4068680 2024-09-15T03:36:45.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.663+0000 7f49b9ffb700 1 -- 172.21.15.110:0/3782920615 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b400fa30 con 0x7f49c4068680 2024-09-15T03:36:45.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.663+0000 7f49cbdea700 1 -- 172.21.15.110:0/3782920615 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49c000f610 con 0x7f49c4068680 2024-09-15T03:36:45.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.667+0000 7f49cbdea700 1 -- 172.21.15.110:0/3782920615 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49c406cc70 con 0x7f49c4068680 2024-09-15T03:36:45.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.667+0000 7f49b9ffb700 1 -- 172.21.15.110:0/3782920615 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) v1 ==== 85847+0+0 (secure 0 0 0) 0x7f49b401a400 con 0x7f49c4068680 2024-09-15T03:36:45.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.667+0000 7f49b9ffb700 1 --2- 172.21.15.110:0/3782920615 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f49b0067c30 0x7f49b006a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:45.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.667+0000 7f49b9ffb700 1 -- 172.21.15.110:0/3782920615 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5262+0+0 (secure 0 0 0) 0x7f49b4084e80 con 0x7f49c4068680 2024-09-15T03:36:45.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.667+0000 7f49c9385700 1 --2- 172.21.15.110:0/3782920615 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f49b0067c30 0x7f49b006a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:45.706 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.667+0000 7f49c9385700 1 --2- 172.21.15.110:0/3782920615 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f49b0067c30 0x7f49b006a0e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f49bc009fd0 tx=0x7f49bc015040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:45.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.671+0000 7f01adffb700 1 -- 172.21.15.110:0/1924997697 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a8015440 con 0x7f01b00ad010 2024-09-15T03:36:45.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.671+0000 7fcf5affd700 1 -- 172.21.15.110:0/26495045 <== mon.2 v2:172.21.15.110:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf44007ad0 con 0x7fcf48004810 2024-09-15T03:36:45.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.671+0000 7fcf5affd700 1 -- 172.21.15.110:0/26495045 <== mon.2 v2:172.21.15.110:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf44006820 con 0x7fcf48004810 2024-09-15T03:36:45.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.671+0000 7fcf5affd700 1 -- 172.21.15.110:0/26495045 <== mon.2 v2:172.21.15.110:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf44006bc0 con 0x7fcf48004810 2024-09-15T03:36:45.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.671+0000 7fe4a816a700 1 -- 172.21.15.110:0/3023401115 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4a00a4620 con 0x7fe4a013d120 2024-09-15T03:36:45.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.671+0000 7f01adffb700 1 -- 172.21.15.110:0/1924997697 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01a80155e0 con 0x7f01b00ad010 2024-09-15T03:36:45.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.671+0000 7fcf62286700 1 -- 172.21.15.110:0/26495045 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fcf48004810 msgr2=0x7fcf48004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:45.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.671+0000 7fcf62286700 1 --2- 172.21.15.110:0/26495045 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fcf48004810 0x7fcf48004c80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fcf4400e600 tx=0x7fcf44002e70 comp rx=0 tx=0).stop 2024-09-15T03:36:45.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.671+0000 7fcf62286700 1 -- 172.21.15.110:0/26495045 shutdown_connections 2024-09-15T03:36:45.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.671+0000 7fcf62286700 1 --2- 172.21.15.110:0/26495045 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fcf48004810 0x7fcf48004c80 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.671+0000 7fcf62286700 1 -- 172.21.15.110:0/26495045 >> 172.21.15.110:0/26495045 conn(0x7fcf5c067670 msgr2=0x7fcf5c067a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:45.709 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.671+0000 7f01adffb700 1 -- 172.21.15.110:0/1924997697 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a80159a0 con 0x7f01b00ad010 2024-09-15T03:36:45.709 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.671+0000 7f01b7afc700 1 -- 172.21.15.110:0/1924997697 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01a0009420 con 0x7f01b00ad010 2024-09-15T03:36:45.709 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.671+0000 7f01b7afc700 1 -- 172.21.15.110:0/1924997697 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01b0121ef0 con 0x7f01b00ad010 2024-09-15T03:36:45.709 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.671+0000 7fe495ffb700 1 -- 172.21.15.110:0/3023401115 <== mon.2 v2:172.21.15.110:3301/0 4 ==== mgrmap(e 5) v1 ==== 85847+0+0 (secure 0 0 0) 0x7fe498023030 con 0x7fe4a013d120 2024-09-15T03:36:45.709 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.671+0000 7fcf62286700 1 -- 172.21.15.110:0/26495045 shutdown_connections 2024-09-15T03:36:45.709 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.671+0000 7fcf62286700 1 -- 172.21.15.110:0/26495045 wait complete. 2024-09-15T03:36:45.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.671+0000 7f01adffb700 1 -- 172.21.15.110:0/1924997697 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) v1 ==== 85847+0+0 (secure 0 0 0) 0x7f01a8015d50 con 0x7f01b00ad010 2024-09-15T03:36:45.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.671+0000 7fe495ffb700 1 --2- 172.21.15.110:0/3023401115 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fe484067b50 0x7fe48406a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:45.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.671+0000 7fe4a5f06700 1 --2- 172.21.15.110:0/3023401115 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fe484067b50 0x7fe48406a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:45.710 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.671+0000 7fe495ffb700 1 -- 172.21.15.110:0/3023401115 <== mon.2 v2:172.21.15.110:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5262+0+0 (secure 0 0 0) 0x7fe4980877b0 con 0x7fe4a013d120 2024-09-15T03:36:45.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.675+0000 7f01adffb700 1 --2- 172.21.15.110:0/1924997697 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f019c067b50 0x7f019c06a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:45.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.675+0000 7f01adffb700 1 -- 172.21.15.110:0/1924997697 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5262+0+0 (secure 0 0 0) 0x7f01a8085c70 con 0x7f01b00ad010 2024-09-15T03:36:45.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.675+0000 7fe4a5f06700 1 --2- 172.21.15.110:0/3023401115 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fe484067b50 0x7fe48406a000 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe49c0097f0 tx=0x7fe49c009ae0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:45.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.675+0000 7f01b5898700 1 --2- 172.21.15.110:0/1924997697 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f019c067b50 0x7f019c06a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:45.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.675+0000 7fcf62286700 1 Processor -- start 2024-09-15T03:36:45.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.675+0000 7fcf62286700 1 -- start start 2024-09-15T03:36:45.712 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.675+0000 7f01b5898700 1 --2- 172.21.15.110:0/1924997697 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f019c067b50 0x7f019c06a000 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f01a4006d40 tx=0x7f01a4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:45.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.675+0000 7fcf62286700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf48004810 0x7fcf5c133610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:45.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.675+0000 7fcf62286700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf5c0ad920 0x7fcf5c148710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:45.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.675+0000 7fcf62286700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fcf5c0ae2d0 0x7fcf5c148c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:45.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.675+0000 7fcf62286700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5c0ae720 con 0x7fcf5c0ad920 2024-09-15T03:36:45.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.675+0000 7fcf62286700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5c0af5e0 con 0x7fcf48004810 2024-09-15T03:36:45.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.675+0000 7fcf62286700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5c0af8c0 con 0x7fcf5c0ae2d0 2024-09-15T03:36:45.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.675+0000 7fcf60823700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fcf5c0ae2d0 0x7fcf5c148c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:45.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.675+0000 7fcf60823700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fcf5c0ae2d0 0x7fcf5c148c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3301/0 says I am v2:172.21.15.110:38852/0 (socket says 172.21.15.110:38852) 2024-09-15T03:36:45.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.675+0000 7fcf60823700 1 -- 172.21.15.110:0/2557497809 learned_addr learned my addr 172.21.15.110:0/2557497809 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:36:45.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.675+0000 7fcf5b7fe700 1 --2- 172.21.15.110:0/2557497809 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf5c0ad920 0x7fcf5c148710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:45.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.675+0000 7fcf5bfff700 1 --2- 172.21.15.110:0/2557497809 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf48004810 0x7fcf5c133610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:45.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.675+0000 7fcf60823700 1 -- 172.21.15.110:0/2557497809 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf48004810 msgr2=0x7fcf5c133610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:45.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.675+0000 7fcf60823700 1 --2- 172.21.15.110:0/2557497809 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf48004810 0x7fcf5c133610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.675+0000 7fcf60823700 1 -- 172.21.15.110:0/2557497809 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf5c0ad920 msgr2=0x7fcf5c148710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:45.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.675+0000 7fcf60823700 1 --2- 172.21.15.110:0/2557497809 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf5c0ad920 0x7fcf5c148710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.675+0000 7fcf60823700 1 -- 172.21.15.110:0/2557497809 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf5c0addb0 con 0x7fcf5c0ae2d0 2024-09-15T03:36:45.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.675+0000 7fcf60823700 1 --2- 172.21.15.110:0/2557497809 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fcf5c0ae2d0 0x7fcf5c148c50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c004c10 tx=0x7fcf4c005010 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:45.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.679+0000 7f49cbdea700 1 -- 172.21.15.110:0/3782920615 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49a40050b0 con 0x7f49c4068680 2024-09-15T03:36:45.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.679+0000 7f01977fe700 1 -- 172.21.15.110:0/1924997697 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01b0064bd0 con 0x7f01b00ad010 2024-09-15T03:36:45.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.683+0000 7fcf53fff700 1 -- 172.21.15.110:0/2557497809 <== mon.2 v2:172.21.15.110:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c0115f0 con 0x7fcf5c0ae2d0 2024-09-15T03:36:45.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.683+0000 7fcf53fff700 1 -- 172.21.15.110:0/2557497809 <== mon.2 v2:172.21.15.110:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf4c011770 con 0x7fcf5c0ae2d0 2024-09-15T03:36:45.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.683+0000 7fcf62286700 1 -- 172.21.15.110:0/2557497809 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf5c14bca0 con 0x7fcf5c0ae2d0 2024-09-15T03:36:45.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.683+0000 7fcf62286700 1 -- 172.21.15.110:0/2557497809 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf5c14c0f0 con 0x7fcf5c0ae2d0 2024-09-15T03:36:45.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.683+0000 7fcf53fff700 1 -- 172.21.15.110:0/2557497809 <== mon.2 v2:172.21.15.110:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c011b70 con 0x7fcf5c0ae2d0 2024-09-15T03:36:45.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.683+0000 7fe495ffb700 1 -- 172.21.15.110:0/3023401115 <== mon.2 v2:172.21.15.110:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fe498054ad0 con 0x7fe4a013d120 2024-09-15T03:36:45.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.687+0000 7f01adffb700 1 -- 172.21.15.110:0/1924997697 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f01a804d480 con 0x7f01b00ad010 2024-09-15T03:36:45.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.695+0000 7fcf53fff700 1 -- 172.21.15.110:0/2557497809 <== mon.2 v2:172.21.15.110:3301/0 4 ==== mgrmap(e 5) v1 ==== 85847+0+0 (secure 0 0 0) 0x7fcf4c0143e0 con 0x7fcf5c0ae2d0 2024-09-15T03:36:45.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.695+0000 7fcf53fff700 1 --2- 172.21.15.110:0/2557497809 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fcf4806dc20 0x7fcf480700d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:45.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.695+0000 7fcf5bfff700 1 --2- 172.21.15.110:0/2557497809 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fcf4806dc20 0x7fcf480700d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:45.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.695+0000 7f49b9ffb700 1 -- 172.21.15.110:0/3782920615 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f49b40521c0 con 0x7f49c4068680 2024-09-15T03:36:45.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.695+0000 7fcf53fff700 1 -- 172.21.15.110:0/2557497809 <== mon.2 v2:172.21.15.110:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5262+0+0 (secure 0 0 0) 0x7fcf4c084cb0 con 0x7fcf5c0ae2d0 2024-09-15T03:36:45.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.695+0000 7fcf5bfff700 1 --2- 172.21.15.110:0/2557497809 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fcf4806dc20 0x7fcf480700d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c0672c0 tx=0x7fcf54008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:45.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.703+0000 7fcf62286700 1 -- 172.21.15.110:0/2557497809 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf400050b0 con 0x7fcf5c0ae2d0 2024-09-15T03:36:45.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.715+0000 7fcf53fff700 1 -- 172.21.15.110:0/2557497809 <== mon.2 v2:172.21.15.110:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fcf4c051ed0 con 0x7fcf5c0ae2d0 2024-09-15T03:36:45.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.907+0000 7ff0c9765700 1 -- 172.21.15.110:0/517893745 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7ff0a0005630 con 0x7ff0c40aced0 2024-09-15T03:36:45.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.907+0000 7ff0b2ffd700 1 -- 172.21.15.110:0/517893745 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff0b8015090 con 0x7ff0c40aced0 2024-09-15T03:36:45.913 INFO:teuthology.orchestra.run.smithi110.stdout:60129542147 2024-09-15T03:36:45.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.911+0000 7ff0b0ff9700 1 -- 172.21.15.110:0/517893745 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7ff0a8067a80 msgr2=0x7ff0a8069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:45.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.911+0000 7ff0b0ff9700 1 --2- 172.21.15.110:0/517893745 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7ff0a8067a80 0x7ff0a8069f30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff0c40a4ca0 tx=0x7ff0b4000bc0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.911+0000 7ff0b0ff9700 1 -- 172.21.15.110:0/517893745 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff0c40aced0 msgr2=0x7ff0c4151410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:45.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.911+0000 7ff0b0ff9700 1 --2- 172.21.15.110:0/517893745 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff0c40aced0 0x7ff0c4151410 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff0b800c830 tx=0x7ff0b8008040 comp rx=0 tx=0).stop 2024-09-15T03:36:45.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.911+0000 7ff0b0ff9700 1 -- 172.21.15.110:0/517893745 shutdown_connections 2024-09-15T03:36:45.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.911+0000 7ff0b0ff9700 1 --2- 172.21.15.110:0/517893745 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7ff0a8067a80 0x7ff0a8069f30 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.911+0000 7ff0b0ff9700 1 --2- 172.21.15.110:0/517893745 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff0c413d120 0x7ff0c4138370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.911+0000 7ff0b0ff9700 1 --2- 172.21.15.110:0/517893745 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7ff0c4139bc0 0x7ff0c4137780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.911+0000 7ff0b0ff9700 1 --2- 172.21.15.110:0/517893745 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff0c40aced0 0x7ff0c4151410 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.911+0000 7ff0b0ff9700 1 -- 172.21.15.110:0/517893745 >> 172.21.15.110:0/517893745 conn(0x7ff0c4067670 msgr2=0x7ff0c40afd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:45.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.911+0000 7ff0b0ff9700 1 -- 172.21.15.110:0/517893745 shutdown_connections 2024-09-15T03:36:45.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.911+0000 7ff0b0ff9700 1 -- 172.21.15.110:0/517893745 wait complete. 2024-09-15T03:36:45.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.919+0000 7f691bc5d700 1 -- 172.21.15.110:0/2802668666 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f68f4005630 con 0x7f69140aced0 2024-09-15T03:36:45.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.919+0000 7f6909ffb700 1 -- 172.21.15.110:0/2802668666 <== mon.2 v2:172.21.15.110:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f69100590e0 con 0x7f69140aced0 2024-09-15T03:36:45.922 INFO:teuthology.orchestra.run.smithi110.stdout:60129542147 2024-09-15T03:36:45.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.919+0000 7f68ff7fe700 1 -- 172.21.15.110:0/2802668666 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f68f8067b80 msgr2=0x7f68f806a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:45.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.919+0000 7f68ff7fe700 1 --2- 172.21.15.110:0/2802668666 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f68f8067b80 0x7f68f806a030 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f69140a7290 tx=0x7f6904008040 comp rx=0 tx=0).stop 2024-09-15T03:36:45.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.923+0000 7f68ff7fe700 1 -- 172.21.15.110:0/2802668666 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f69140aced0 msgr2=0x7f69140a79e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:45.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.923+0000 7f68ff7fe700 1 --2- 172.21.15.110:0/2802668666 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f69140aced0 0x7f69140a79e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f691000b600 tx=0x7f69100091a0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.923+0000 7f68ff7fe700 1 -- 172.21.15.110:0/2802668666 shutdown_connections 2024-09-15T03:36:45.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.923+0000 7f68ff7fe700 1 --2- 172.21.15.110:0/2802668666 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f68f8067b80 0x7f68f806a030 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.923+0000 7f68ff7fe700 1 --2- 172.21.15.110:0/2802668666 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f691414cb20 0x7f69140a8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.923+0000 7f68ff7fe700 1 --2- 172.21.15.110:0/2802668666 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f691413d120 0x7f691414c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.923+0000 7f68ff7fe700 1 --2- 172.21.15.110:0/2802668666 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f69140aced0 0x7f69140a79e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.923+0000 7f68ff7fe700 1 -- 172.21.15.110:0/2802668666 >> 172.21.15.110:0/2802668666 conn(0x7f6914067670 msgr2=0x7f691406ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:45.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.923+0000 7f68ff7fe700 1 -- 172.21.15.110:0/2802668666 shutdown_connections 2024-09-15T03:36:45.927 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.923+0000 7f68ff7fe700 1 -- 172.21.15.110:0/2802668666 wait complete. 2024-09-15T03:36:45.929 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542147 got 60129542147 for osd.5 2024-09-15T03:36:45.929 DEBUG:teuthology.parallel:result is None 2024-09-15T03:36:45.940 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542147 got 60129542147 for osd.3 2024-09-15T03:36:45.941 DEBUG:teuthology.parallel:result is None 2024-09-15T03:36:45.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.951+0000 7f9570468700 1 -- 172.21.15.110:0/3418945193 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f95680a6fe0 con 0x7f956806be40 2024-09-15T03:36:45.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.951+0000 7f955e7fc700 1 -- 172.21.15.110:0/3418945193 <== mon.2 v2:172.21.15.110:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f95680a6fe0 con 0x7f956806be40 2024-09-15T03:36:45.955 INFO:teuthology.orchestra.run.smithi110.stdout:55834574852 2024-09-15T03:36:45.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.951+0000 7f9570468700 1 -- 172.21.15.110:0/3418945193 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f954c067b50 msgr2=0x7f954c06a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:45.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.951+0000 7f9570468700 1 --2- 172.21.15.110:0/3418945193 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f954c067b50 0x7f954c06a000 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f95680a5550 tx=0x7f95640091a0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.951+0000 7f9570468700 1 -- 172.21.15.110:0/3418945193 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f956806be40 msgr2=0x7f9568146270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:45.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.951+0000 7f9570468700 1 --2- 172.21.15.110:0/3418945193 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f956806be40 0x7f9568146270 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f955800c5d0 tx=0x7f9558007f60 comp rx=0 tx=0).stop 2024-09-15T03:36:45.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.951+0000 7f9570468700 1 -- 172.21.15.110:0/3418945193 shutdown_connections 2024-09-15T03:36:45.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.951+0000 7f9570468700 1 --2- 172.21.15.110:0/3418945193 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f954c067b50 0x7f954c06a000 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.951+0000 7f9570468700 1 --2- 172.21.15.110:0/3418945193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9568145400 0x7f95680a72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.951+0000 7f9570468700 1 --2- 172.21.15.110:0/3418945193 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95680acd80 0x7f95681336b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.951+0000 7f9570468700 1 --2- 172.21.15.110:0/3418945193 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f956806be40 0x7f9568146270 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.951+0000 7f9570468700 1 -- 172.21.15.110:0/3418945193 >> 172.21.15.110:0/3418945193 conn(0x7f9568067670 msgr2=0x7f95680ae0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:45.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.951+0000 7f9570468700 1 -- 172.21.15.110:0/3418945193 shutdown_connections 2024-09-15T03:36:45.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.951+0000 7f9570468700 1 -- 172.21.15.110:0/3418945193 wait complete. 2024-09-15T03:36:45.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.951+0000 7fb7adeab700 1 -- 172.21.15.110:0/418402908 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fb7a80ae3e0 con 0x7fb7a8139bc0 2024-09-15T03:36:45.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.951+0000 7fb78f7fe700 1 -- 172.21.15.110:0/418402908 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb79c055eb0 con 0x7fb7a8139bc0 2024-09-15T03:36:45.958 INFO:teuthology.orchestra.run.smithi110.stdout:60129542148 2024-09-15T03:36:45.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.955+0000 7fb7adeab700 1 -- 172.21.15.110:0/418402908 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fb794067b50 msgr2=0x7fb79406a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:45.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.955+0000 7fb7adeab700 1 --2- 172.21.15.110:0/418402908 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fb794067b50 0x7fb79406a000 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb798009d40 tx=0x7fb798008040 comp rx=0 tx=0).stop 2024-09-15T03:36:45.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.955+0000 7fb7adeab700 1 -- 172.21.15.110:0/418402908 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7a8139bc0 msgr2=0x7fb7a80ae730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:45.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.955+0000 7fb7adeab700 1 --2- 172.21.15.110:0/418402908 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7a8139bc0 0x7fb7a80ae730 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb79c00b610 tx=0x7fb79c0090f0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.955+0000 7fb7adeab700 1 -- 172.21.15.110:0/418402908 shutdown_connections 2024-09-15T03:36:45.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.955+0000 7fb7adeab700 1 --2- 172.21.15.110:0/418402908 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fb794067b50 0x7fb79406a000 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.955+0000 7fb7adeab700 1 --2- 172.21.15.110:0/418402908 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fb7a813d120 0x7fb7a80a4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.955+0000 7fb7adeab700 1 --2- 172.21.15.110:0/418402908 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7a8139bc0 0x7fb7a80ae730 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.955+0000 7fb7adeab700 1 --2- 172.21.15.110:0/418402908 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7a80aced0 0x7fb7a8137660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.955+0000 7fb7adeab700 1 -- 172.21.15.110:0/418402908 >> 172.21.15.110:0/418402908 conn(0x7fb7a8067670 msgr2=0x7fb7a8138200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:45.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.955+0000 7fb7adeab700 1 -- 172.21.15.110:0/418402908 shutdown_connections 2024-09-15T03:36:45.962 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.955+0000 7fb7adeab700 1 -- 172.21.15.110:0/418402908 wait complete. 2024-09-15T03:36:45.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.959+0000 7f01977fe700 1 -- 172.21.15.110:0/1924997697 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f01b0121690 con 0x7f01b00ad010 2024-09-15T03:36:45.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.959+0000 7f01adffb700 1 -- 172.21.15.110:0/1924997697 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f01a801a080 con 0x7f01b00ad010 2024-09-15T03:36:45.964 INFO:teuthology.orchestra.run.smithi110.stdout:60129542148 2024-09-15T03:36:45.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.963+0000 7f01977fe700 1 -- 172.21.15.110:0/1924997697 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f019c067b50 msgr2=0x7f019c06a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:45.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.963+0000 7f01977fe700 1 --2- 172.21.15.110:0/1924997697 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f019c067b50 0x7f019c06a000 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f01a4006d40 tx=0x7f01a4008040 comp rx=0 tx=0).stop 2024-09-15T03:36:45.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.963+0000 7f01977fe700 1 -- 172.21.15.110:0/1924997697 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f01b00ad010 msgr2=0x7f01b0121930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:45.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.963+0000 7f01977fe700 1 --2- 172.21.15.110:0/1924997697 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f01b00ad010 0x7f01b0121930 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f01a800cf30 tx=0x7f01a8007840 comp rx=0 tx=0).stop 2024-09-15T03:36:45.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.963+0000 7f01977fe700 1 -- 172.21.15.110:0/1924997697 shutdown_connections 2024-09-15T03:36:45.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.963+0000 7f01977fe700 1 --2- 172.21.15.110:0/1924997697 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f019c067b50 0x7f019c06a000 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.963+0000 7f01977fe700 1 --2- 172.21.15.110:0/1924997697 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f01b00ad010 0x7f01b0121930 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.963+0000 7f01977fe700 1 --2- 172.21.15.110:0/1924997697 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01b00acc40 0x7f01b0120680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.963+0000 7f01977fe700 1 --2- 172.21.15.110:0/1924997697 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f01b00ac3c0 0x7f01b00a7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.963+0000 7f01977fe700 1 -- 172.21.15.110:0/1924997697 >> 172.21.15.110:0/1924997697 conn(0x7f01b0067810 msgr2=0x7f01b00b02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:45.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.963+0000 7f01977fe700 1 -- 172.21.15.110:0/1924997697 shutdown_connections 2024-09-15T03:36:45.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.963+0000 7f01977fe700 1 -- 172.21.15.110:0/1924997697 wait complete. 2024-09-15T03:36:45.969 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.4 2024-09-15T03:36:45.969 DEBUG:teuthology.parallel:result is None 2024-09-15T03:36:45.975 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542148 for osd.7 2024-09-15T03:36:45.976 DEBUG:teuthology.parallel:result is None 2024-09-15T03:36:45.976 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542148 for osd.2 2024-09-15T03:36:45.976 DEBUG:teuthology.parallel:result is None 2024-09-15T03:36:45.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.987+0000 7fe4a816a700 1 -- 172.21.15.110:0/3023401115 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fe4a00a5ae0 con 0x7fe4a013d120 2024-09-15T03:36:45.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.987+0000 7fe495ffb700 1 -- 172.21.15.110:0/3023401115 <== mon.2 v2:172.21.15.110:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe498024e80 con 0x7fe4a013d120 2024-09-15T03:36:45.993 INFO:teuthology.orchestra.run.smithi110.stdout:60129542147 2024-09-15T03:36:45.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.991+0000 7fe4a816a700 1 -- 172.21.15.110:0/3023401115 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fe484067b50 msgr2=0x7fe48406a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:45.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.991+0000 7fe4a816a700 1 --2- 172.21.15.110:0/3023401115 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fe484067b50 0x7fe48406a000 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe49c0097f0 tx=0x7fe49c009ae0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.991+0000 7fe4a816a700 1 -- 172.21.15.110:0/3023401115 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fe4a013d120 msgr2=0x7fe4a0154ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:45.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.991+0000 7fe4a816a700 1 --2- 172.21.15.110:0/3023401115 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fe4a013d120 0x7fe4a0154ce0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe4a0069660 tx=0x7fe498019000 comp rx=0 tx=0).stop 2024-09-15T03:36:45.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.991+0000 7fe4a816a700 1 -- 172.21.15.110:0/3023401115 shutdown_connections 2024-09-15T03:36:45.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.991+0000 7fe4a816a700 1 --2- 172.21.15.110:0/3023401115 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fe484067b50 0x7fe48406a000 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.991+0000 7fe4a816a700 1 --2- 172.21.15.110:0/3023401115 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fe4a013d120 0x7fe4a0154ce0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.991+0000 7fe4a816a700 1 --2- 172.21.15.110:0/3023401115 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4a00aced0 0x7fe4a01517d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.991+0000 7fe4a816a700 1 --2- 172.21.15.110:0/3023401115 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe4a00aa7a0 0x7fe4a00aed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:45.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.991+0000 7fe4a816a700 1 -- 172.21.15.110:0/3023401115 >> 172.21.15.110:0/3023401115 conn(0x7fe4a0067670 msgr2=0x7fe4a0071440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:45.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.991+0000 7fe4a816a700 1 -- 172.21.15.110:0/3023401115 shutdown_connections 2024-09-15T03:36:45.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.991+0000 7fe4a816a700 1 -- 172.21.15.110:0/3023401115 wait complete. 2024-09-15T03:36:45.998 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.995+0000 7f49cbdea700 1 -- 172.21.15.110:0/3782920615 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f49a4005630 con 0x7f49c4068680 2024-09-15T03:36:45.998 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.995+0000 7f49b9ffb700 1 -- 172.21.15.110:0/3782920615 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f49b4055240 con 0x7f49c4068680 2024-09-15T03:36:45.998 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.995+0000 7fcf62286700 1 -- 172.21.15.110:0/2557497809 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fcf40005360 con 0x7fcf5c0ae2d0 2024-09-15T03:36:45.999 INFO:teuthology.orchestra.run.smithi110.stdout:55834574851 2024-09-15T03:36:45.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.995+0000 7fcf53fff700 1 -- 172.21.15.110:0/2557497809 <== mon.2 v2:172.21.15.110:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fcf4c054f50 con 0x7fcf5c0ae2d0 2024-09-15T03:36:45.999 INFO:teuthology.orchestra.run.smithi110.stdout:55834574852 2024-09-15T03:36:46.001 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.995+0000 7f49cbdea700 1 -- 172.21.15.110:0/3782920615 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f49b0067c30 msgr2=0x7f49b006a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:46.001 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.995+0000 7f49cbdea700 1 --2- 172.21.15.110:0/3782920615 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f49b0067c30 0x7f49b006a0e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f49bc009fd0 tx=0x7f49bc015040 comp rx=0 tx=0).stop 2024-09-15T03:36:46.001 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.995+0000 7f49cbdea700 1 -- 172.21.15.110:0/3782920615 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49c4068680 msgr2=0x7f49c413b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:46.001 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.995+0000 7f49cbdea700 1 --2- 172.21.15.110:0/3782920615 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49c4068680 0x7f49c413b3c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f49b400b810 tx=0x7f49b40091d0 comp rx=0 tx=0).stop 2024-09-15T03:36:46.001 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.995+0000 7f49cbdea700 1 -- 172.21.15.110:0/3782920615 shutdown_connections 2024-09-15T03:36:46.001 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.995+0000 7f49cbdea700 1 --2- 172.21.15.110:0/3782920615 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f49b0067c30 0x7f49b006a0e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:46.001 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.995+0000 7f49cbdea700 1 --2- 172.21.15.110:0/3782920615 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f49c4138bc0 0x7f49c4134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:46.001 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.995+0000 7f49cbdea700 1 --2- 172.21.15.110:0/3782920615 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49c4068a50 0x7f49c406c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:46.001 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.995+0000 7f49cbdea700 1 --2- 172.21.15.110:0/3782920615 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49c4068680 0x7f49c413b3c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:46.001 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.995+0000 7f49cbdea700 1 -- 172.21.15.110:0/3782920615 >> 172.21.15.110:0/3782920615 conn(0x7f49c40ad150 msgr2=0x7f49c4137e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:46.001 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.995+0000 7f49cbdea700 1 -- 172.21.15.110:0/3782920615 shutdown_connections 2024-09-15T03:36:46.002 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.995+0000 7f49cbdea700 1 -- 172.21.15.110:0/3782920615 wait complete. 2024-09-15T03:36:46.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.995+0000 7fcf62286700 1 -- 172.21.15.110:0/2557497809 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fcf4806dc20 msgr2=0x7fcf480700d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:46.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.995+0000 7fcf62286700 1 --2- 172.21.15.110:0/2557497809 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fcf4806dc20 0x7fcf480700d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c0672c0 tx=0x7fcf54008040 comp rx=0 tx=0).stop 2024-09-15T03:36:46.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.995+0000 7fcf62286700 1 -- 172.21.15.110:0/2557497809 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fcf5c0ae2d0 msgr2=0x7fcf5c148c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:46.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.995+0000 7fcf62286700 1 --2- 172.21.15.110:0/2557497809 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fcf5c0ae2d0 0x7fcf5c148c50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c004c10 tx=0x7fcf4c005010 comp rx=0 tx=0).stop 2024-09-15T03:36:46.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.995+0000 7fcf62286700 1 -- 172.21.15.110:0/2557497809 shutdown_connections 2024-09-15T03:36:46.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.995+0000 7fcf62286700 1 --2- 172.21.15.110:0/2557497809 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fcf4806dc20 0x7fcf480700d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:46.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.995+0000 7fcf62286700 1 --2- 172.21.15.110:0/2557497809 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fcf5c0ae2d0 0x7fcf5c148c50 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:46.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.995+0000 7fcf62286700 1 --2- 172.21.15.110:0/2557497809 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf5c0ad920 0x7fcf5c148710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:46.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.995+0000 7fcf62286700 1 --2- 172.21.15.110:0/2557497809 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf48004810 0x7fcf5c133610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:46.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.995+0000 7fcf62286700 1 -- 172.21.15.110:0/2557497809 >> 172.21.15.110:0/2557497809 conn(0x7fcf5c067670 msgr2=0x7fcf5c070e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:46.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.995+0000 7fcf62286700 1 -- 172.21.15.110:0/2557497809 shutdown_connections 2024-09-15T03:36:46.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:45.995+0000 7fcf62286700 1 -- 172.21.15.110:0/2557497809 wait complete. 2024-09-15T03:36:46.011 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.1 2024-09-15T03:36:46.012 DEBUG:teuthology.parallel:result is None 2024-09-15T03:36:46.012 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542147 got 60129542147 for osd.6 2024-09-15T03:36:46.013 DEBUG:teuthology.parallel:result is None 2024-09-15T03:36:46.013 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.0 2024-09-15T03:36:46.013 DEBUG:teuthology.parallel:result is None 2024-09-15T03:36:46.014 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-15T03:36:46.014 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T03:36:46.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.099+0000 7fbcb4058700 1 Processor -- start 2024-09-15T03:36:46.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.099+0000 7fbcb4058700 1 -- start start 2024-09-15T03:36:46.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.103+0000 7fbcb4058700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fbcac0a44d0 0x7fbcac0aeae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:46.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.103+0000 7fbcb4058700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fbcac13a2f0 0x7fbcac13a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:46.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.103+0000 7fbcb4058700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcac0a4b40 con 0x7fbcac140fa0 2024-09-15T03:36:46.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.103+0000 7fbcb4058700 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcac141370 con 0x7fbcac13a2f0 2024-09-15T03:36:46.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.103+0000 7fbcb4058700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcac0a6510 con 0x7fbcac0a44d0 2024-09-15T03:36:46.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.103+0000 7fbcb15f3700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fbcac0a44d0 0x7fbcac0aeae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:46.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.103+0000 7fbcb25f5700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fbcac13a2f0 0x7fbcac13a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:46.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.103+0000 7fbcb15f3700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fbcac0a44d0 0x7fbcac0aeae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3301/0 says I am v2:172.21.15.110:38862/0 (socket says 172.21.15.110:38862) 2024-09-15T03:36:46.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.103+0000 7fbcb15f3700 1 -- 172.21.15.110:0/2720573444 learned_addr learned my addr 172.21.15.110:0/2720573444 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:36:46.109 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.103+0000 7fbcb15f3700 1 -- 172.21.15.110:0/2720573444 >> v2:172.21.15.110:3300/0 conn(0x7fbcac13a2f0 msgr2=0x7fbcac13a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:36:46.109 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.103+0000 7fbcb15f3700 1 --2- 172.21.15.110:0/2720573444 >> v2:172.21.15.110:3300/0 conn(0x7fbcac13a2f0 0x7fbcac13a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:46.109 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.103+0000 7fbcb15f3700 1 -- 172.21.15.110:0/2720573444 >> v1:172.21.15.177:6789/0 conn(0x7fbcac140fa0 legacy=0x7fbcac141700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:46.109 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.103+0000 7fbcb15f3700 1 -- 172.21.15.110:0/2720573444 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcac0a67f0 con 0x7fbcac0a44d0 2024-09-15T03:36:46.109 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.103+0000 7fbcb25f5700 1 --2- 172.21.15.110:0/2720573444 >> v2:172.21.15.110:3300/0 conn(0x7fbcac13a2f0 0x7fbcac13a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T03:36:46.109 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.103+0000 7fbcb15f3700 1 --2- 172.21.15.110:0/2720573444 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fbcac0a44d0 0x7fbcac0aeae0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbca80099b0 tx=0x7fbca8009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cffbe0f702f0d23a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:46.109 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.103+0000 7fbcb0df2700 1 -- 172.21.15.110:0/2720573444 <== mon.2 v2:172.21.15.110:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca8009ea0 con 0x7fbcac0a44d0 2024-09-15T03:36:46.109 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.103+0000 7fbcb0df2700 1 -- 172.21.15.110:0/2720573444 <== mon.2 v2:172.21.15.110:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbca800b860 con 0x7fbcac0a44d0 2024-09-15T03:36:46.109 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.103+0000 7fbcb0df2700 1 -- 172.21.15.110:0/2720573444 <== mon.2 v2:172.21.15.110:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca800bc00 con 0x7fbcac0a44d0 2024-09-15T03:36:46.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.103+0000 7fbcb4058700 1 -- 172.21.15.110:0/2720573444 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fbcac0a44d0 msgr2=0x7fbcac0aeae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:46.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.103+0000 7fbcb4058700 1 --2- 172.21.15.110:0/2720573444 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fbcac0a44d0 0x7fbcac0aeae0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbca80099b0 tx=0x7fbca8009250 comp rx=0 tx=0).stop 2024-09-15T03:36:46.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.103+0000 7fbcb4058700 1 -- 172.21.15.110:0/2720573444 shutdown_connections 2024-09-15T03:36:46.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.103+0000 7fbcb4058700 1 --2- 172.21.15.110:0/2720573444 >> v2:172.21.15.110:3300/0 conn(0x7fbcac13a2f0 0x7fbcac13a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:46.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.103+0000 7fbcb4058700 1 --2- 172.21.15.110:0/2720573444 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fbcac0a44d0 0x7fbcac0aeae0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:46.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.103+0000 7fbcb4058700 1 -- 172.21.15.110:0/2720573444 >> 172.21.15.110:0/2720573444 conn(0x7fbcac06f830 msgr2=0x7fbcac0b4170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:46.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.103+0000 7fbcb4058700 1 -- 172.21.15.110:0/2720573444 shutdown_connections 2024-09-15T03:36:46.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.103+0000 7fbcb4058700 1 -- 172.21.15.110:0/2720573444 wait complete. 2024-09-15T03:36:46.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.103+0000 7fbcb4058700 1 Processor -- start 2024-09-15T03:36:46.113 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.103+0000 7fbcb4058700 1 -- start start 2024-09-15T03:36:46.113 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.107+0000 7fbcb4058700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fbcac0a44d0 0x7fbcac137f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:46.113 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.107+0000 7fbcb4058700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbcac13a2f0 0x7fbcac150760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:46.113 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.107+0000 7fbcb4058700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbcac140fa0 0x7fbcac150ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:46.113 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.107+0000 7fbcb4058700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcac0a9d70 con 0x7fbcac140fa0 2024-09-15T03:36:46.113 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.107+0000 7fbcb4058700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcac0ae290 con 0x7fbcac13a2f0 2024-09-15T03:36:46.113 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.107+0000 7fbcb4058700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcac0a8ec0 con 0x7fbcac0a44d0 2024-09-15T03:36:46.113 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.107+0000 7fbcb1df4700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fbcac0a44d0 0x7fbcac137f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:46.113 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.107+0000 7fbcb1df4700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fbcac0a44d0 0x7fbcac137f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3301/0 says I am v2:172.21.15.110:38876/0 (socket says 172.21.15.110:38876) 2024-09-15T03:36:46.113 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.107+0000 7fbcb1df4700 1 -- 172.21.15.110:0/4177461949 learned_addr learned my addr 172.21.15.110:0/4177461949 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:36:46.113 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.107+0000 7fbcb25f5700 1 --2- 172.21.15.110:0/4177461949 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbcac140fa0 0x7fbcac150ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:46.113 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.107+0000 7fbcb1df4700 1 -- 172.21.15.110:0/4177461949 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbcac13a2f0 msgr2=0x7fbcac150760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:46.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.107+0000 7fbcb1df4700 1 --2- 172.21.15.110:0/4177461949 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbcac13a2f0 0x7fbcac150760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:46.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.107+0000 7fbcb1df4700 1 -- 172.21.15.110:0/4177461949 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbcac140fa0 msgr2=0x7fbcac150ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:46.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.107+0000 7fbcb1df4700 1 --2- 172.21.15.110:0/4177461949 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbcac140fa0 0x7fbcac150ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:46.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.107+0000 7fbcb1df4700 1 -- 172.21.15.110:0/4177461949 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbca4009a20 con 0x7fbcac0a44d0 2024-09-15T03:36:46.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.107+0000 7fbcb1df4700 1 --2- 172.21.15.110:0/4177461949 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fbcac0a44d0 0x7fbcac137f80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c00c590 tx=0x7fbc9c00c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:46.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.107+0000 7fbca1ffb700 1 -- 172.21.15.110:0/4177461949 <== mon.2 v2:172.21.15.110:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c011590 con 0x7fbcac0a44d0 2024-09-15T03:36:46.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.107+0000 7fbca1ffb700 1 -- 172.21.15.110:0/4177461949 <== mon.2 v2:172.21.15.110:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc9c0116f0 con 0x7fbcac0a44d0 2024-09-15T03:36:46.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.107+0000 7fbcb4058700 1 -- 172.21.15.110:0/4177461949 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbca80094f0 con 0x7fbcac0a44d0 2024-09-15T03:36:46.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.107+0000 7fbca1ffb700 1 -- 172.21.15.110:0/4177461949 <== mon.2 v2:172.21.15.110:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c011ab0 con 0x7fbcac0a44d0 2024-09-15T03:36:46.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.107+0000 7fbcb4058700 1 -- 172.21.15.110:0/4177461949 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcac151400 con 0x7fbcac0a44d0 2024-09-15T03:36:46.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.107+0000 7fbcb4058700 1 -- 172.21.15.110:0/4177461949 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcac064c70 con 0x7fbcac0a44d0 2024-09-15T03:36:46.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.107+0000 7fbca1ffb700 1 -- 172.21.15.110:0/4177461949 <== mon.2 v2:172.21.15.110:3301/0 4 ==== mgrmap(e 5) v1 ==== 85847+0+0 (secure 0 0 0) 0x7fbc9c01b3b0 con 0x7fbcac0a44d0 2024-09-15T03:36:46.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.111+0000 7fbca1ffb700 1 --2- 172.21.15.110:0/4177461949 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fbc90067a80 0x7fbc90069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:46.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.111+0000 7fbcb15f3700 1 --2- 172.21.15.110:0/4177461949 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fbc90067a80 0x7fbc90069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:46.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.111+0000 7fbca1ffb700 1 -- 172.21.15.110:0/4177461949 <== mon.2 v2:172.21.15.110:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5262+0+0 (secure 0 0 0) 0x7fbc9c0849f0 con 0x7fbcac0a44d0 2024-09-15T03:36:46.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.111+0000 7fbcb15f3700 1 --2- 172.21.15.110:0/4177461949 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fbc90067a80 0x7fbc90069f30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbca80099b0 tx=0x7fbca8017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:46.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.119+0000 7fbca1ffb700 1 -- 172.21.15.110:0/4177461949 <== mon.2 v2:172.21.15.110:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fbc9c051cb0 con 0x7fbcac0a44d0 2024-09-15T03:36:46.277 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.271+0000 7fbcb4058700 1 -- 172.21.15.110:0/4177461949 --> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbcac13c8d0 con 0x7fbc90067a80 2024-09-15T03:36:46.277 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.271+0000 7fbca1ffb700 1 -- 172.21.15.110:0/4177461949 <== mgr.4104 v2:172.21.15.110:6832/22190 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48451 (secure 0 0 0) 0x7fbcac13c8d0 con 0x7fbc90067a80 2024-09-15T03:36:46.278 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-15T03:36:46.279 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.275+0000 7fbcb4058700 1 -- 172.21.15.110:0/4177461949 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fbc90067a80 msgr2=0x7fbc90069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:46.279 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.275+0000 7fbcb4058700 1 --2- 172.21.15.110:0/4177461949 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fbc90067a80 0x7fbc90069f30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbca80099b0 tx=0x7fbca8017000 comp rx=0 tx=0).stop 2024-09-15T03:36:46.279 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.275+0000 7fbcb4058700 1 -- 172.21.15.110:0/4177461949 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fbcac0a44d0 msgr2=0x7fbcac137f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:46.279 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.275+0000 7fbcb4058700 1 --2- 172.21.15.110:0/4177461949 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fbcac0a44d0 0x7fbcac137f80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c00c590 tx=0x7fbc9c00c950 comp rx=0 tx=0).stop 2024-09-15T03:36:46.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.275+0000 7fbcb4058700 1 -- 172.21.15.110:0/4177461949 shutdown_connections 2024-09-15T03:36:46.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.275+0000 7fbcb4058700 1 --2- 172.21.15.110:0/4177461949 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fbc90067a80 0x7fbc90069f30 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:46.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.275+0000 7fbcb4058700 1 --2- 172.21.15.110:0/4177461949 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbcac140fa0 0x7fbcac150ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:46.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.275+0000 7fbcb4058700 1 --2- 172.21.15.110:0/4177461949 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbcac13a2f0 0x7fbcac150760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:46.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.275+0000 7fbcb4058700 1 --2- 172.21.15.110:0/4177461949 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fbcac0a44d0 0x7fbcac137f80 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:46.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.275+0000 7fbcb4058700 1 -- 172.21.15.110:0/4177461949 >> 172.21.15.110:0/4177461949 conn(0x7fbcac06f830 msgr2=0x7fbcac0b1450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:46.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.275+0000 7fbcb4058700 1 -- 172.21.15.110:0/4177461949 shutdown_connections 2024-09-15T03:36:46.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.275+0000 7fbcb4058700 1 -- 172.21.15.110:0/4177461949 wait complete. 2024-09-15T03:36:46.281 INFO:teuthology.orchestra.run.smithi110.stderr:dumped all 2024-09-15T03:36:46.290 INFO:teuthology.orchestra.run.smithi110.stdout:{"pg_ready":true,"pg_map":{"version":16,"stamp":"2024-09-15T03:36:44.604064+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2381552,"kb_used_data":2520,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":752593168,"statfs":{"total":773094113280,"available":770655404032,"internally_reserved":0,"allocated":2580480,"data_stored":1559806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-15T03:36:42.814243+0000","last_change":"2024-09-15T03:36:42.814645+0000","last_active":"2024-09-15T03:36:42.814243+0000","last_peered":"2024-09-15T03:36:42.814243+0000","last_clean":"2024-09-15T03:36:42.814243+0000","last_became_active":"2024-09-15T03:36:40.672828+0000","last_became_peered":"2024-09-15T03:36:40.672828+0000","last_unstale":"2024-09-15T03:36:42.814243+0000","last_undegraded":"2024-09-15T03:36:42.814243+0000","last_fullsized":"2024-09-15T03:36:42.814243+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_clean_scrub_stamp":"2024-09-15T03:36:39.639431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T13:40:53.039569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064069099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-15T03:36:42.686802+0000","last_change":"2024-09-15T03:36:42.687267+0000","last_active":"2024-09-15T03:36:42.686802+0000","last_peered":"2024-09-15T03:36:42.686802+0000","last_clean":"2024-09-15T03:36:42.686802+0000","last_became_active":"2024-09-15T03:36:40.671677+0000","last_became_peered":"2024-09-15T03:36:40.671677+0000","last_unstale":"2024-09-15T03:36:42.686802+0000","last_undegraded":"2024-09-15T03:36:42.686802+0000","last_fullsized":"2024-09-15T03:36:42.686802+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_clean_scrub_stamp":"2024-09-15T03:36:39.639431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:55:15.493964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000861089,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-15T03:36:42.853958+0000","last_change":"2024-09-15T03:36:42.854270+0000","last_active":"2024-09-15T03:36:42.853958+0000","last_peered":"2024-09-15T03:36:42.853958+0000","last_clean":"2024-09-15T03:36:42.853958+0000","last_became_active":"2024-09-15T03:36:41.079216+0000","last_became_peered":"2024-09-15T03:36:41.079216+0000","last_unstale":"2024-09-15T03:36:42.853958+0000","last_undegraded":"2024-09-15T03:36:42.853958+0000","last_fullsized":"2024-09-15T03:36:42.853958+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_clean_scrub_stamp":"2024-09-15T03:36:39.639431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:19:37.610607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062568200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-15T03:36:42.686886+0000","last_change":"2024-09-15T03:36:42.687270+0000","last_active":"2024-09-15T03:36:42.686886+0000","last_peered":"2024-09-15T03:36:42.686886+0000","last_clean":"2024-09-15T03:36:42.686886+0000","last_became_active":"2024-09-15T03:36:41.077266+0000","last_became_peered":"2024-09-15T03:36:41.077266+0000","last_unstale":"2024-09-15T03:36:42.686886+0000","last_undegraded":"2024-09-15T03:36:42.686886+0000","last_fullsized":"2024-09-15T03:36:42.686886+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_clean_scrub_stamp":"2024-09-15T03:36:39.639431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:54:58.435053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067514400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"19'2","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-15T03:36:42.695885+0000","last_change":"2024-09-15T03:36:42.695885+0000","last_active":"2024-09-15T03:36:42.695885+0000","last_peered":"2024-09-15T03:36:42.695885+0000","last_clean":"2024-09-15T03:36:42.695885+0000","last_became_active":"2024-09-15T03:36:40.673743+0000","last_became_peered":"2024-09-15T03:36:40.673743+0000","last_unstale":"2024-09-15T03:36:42.695885+0000","last_undegraded":"2024-09-15T03:36:42.695885+0000","last_fullsized":"2024-09-15T03:36:42.695885+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_clean_scrub_stamp":"2024-09-15T03:36:39.639431+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T13:57:31.462981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002159518,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-15T03:36:42.801323+0000","last_change":"2024-09-15T03:36:42.801569+0000","last_active":"2024-09-15T03:36:42.801323+0000","last_peered":"2024-09-15T03:36:42.801323+0000","last_clean":"2024-09-15T03:36:42.801323+0000","last_became_active":"2024-09-15T03:36:40.669392+0000","last_became_peered":"2024-09-15T03:36:40.669392+0000","last_unstale":"2024-09-15T03:36:42.801323+0000","last_undegraded":"2024-09-15T03:36:42.801323+0000","last_fullsized":"2024-09-15T03:36:42.801323+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_clean_scrub_stamp":"2024-09-15T03:36:39.639431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:00:33.102146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059629799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-15T03:36:42.853828+0000","last_change":"2024-09-15T03:36:42.854222+0000","last_active":"2024-09-15T03:36:42.853828+0000","last_peered":"2024-09-15T03:36:42.853828+0000","last_clean":"2024-09-15T03:36:42.853828+0000","last_became_active":"2024-09-15T03:36:40.672416+0000","last_became_peered":"2024-09-15T03:36:40.672416+0000","last_unstale":"2024-09-15T03:36:42.853828+0000","last_undegraded":"2024-09-15T03:36:42.853828+0000","last_fullsized":"2024-09-15T03:36:42.853828+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_clean_scrub_stamp":"2024-09-15T03:36:39.639431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T15:08:21.420921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065187600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"17'1","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-15T03:36:42.694075+0000","last_change":"2024-09-15T03:36:42.694486+0000","last_active":"2024-09-15T03:36:42.694075+0000","last_peered":"2024-09-15T03:36:42.694075+0000","last_clean":"2024-09-15T03:36:42.694075+0000","last_became_active":"2024-09-15T03:36:40.674442+0000","last_became_peered":"2024-09-15T03:36:40.674442+0000","last_unstale":"2024-09-15T03:36:42.694075+0000","last_undegraded":"2024-09-15T03:36:42.694075+0000","last_fullsized":"2024-09-15T03:36:42.694075+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_clean_scrub_stamp":"2024-09-15T03:36:39.639431+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:25:57.474430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067001500000000004,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"16'72","reported_seq":102,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T03:36:42.541403+0000","last_change":"2024-09-15T03:36:38.815652+0000","last_active":"2024-09-15T03:36:42.541403+0000","last_peered":"2024-09-15T03:36:42.541403+0000","last_clean":"2024-09-15T03:36:42.541403+0000","last_became_active":"2024-09-15T03:36:38.813490+0000","last_became_peered":"2024-09-15T03:36:38.813490+0000","last_unstale":"2024-09-15T03:36:42.541403+0000","last_undegraded":"2024-09-15T03:36:42.541403+0000","last_fullsized":"2024-09-15T03:36:42.541403+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:37.618040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:37.618040+0000","last_clean_scrub_stamp":"2024-09-15T03:36:37.618040+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:02:12.095255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542148,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298068,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073772,"statfs":{"total":96636764160,"available":96331542528,"internally_reserved":0,"allocated":720896,"data_stored":591667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97099999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4330000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5089999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.645}]}]},{"osd":6,"up_from":14,"seq":60129542147,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297548,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074292,"statfs":{"total":96636764160,"available":96332075008,"internally_reserved":0,"allocated":188416,"data_stored":62739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2509999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.476}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7210000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.095}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.873}]}]},{"osd":5,"up_from":14,"seq":60129542147,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297548,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074292,"statfs":{"total":96636764160,"available":96332075008,"internally_reserved":0,"allocated":192512,"data_stored":62758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92700000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.048}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.145}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5660000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.484}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.171}]}]},{"osd":3,"up_from":14,"seq":60129542147,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297548,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074292,"statfs":{"total":96636764160,"available":96332075008,"internally_reserved":0,"allocated":188416,"data_stored":62739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.05}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.085}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.157}]}]},{"osd":2,"up_from":14,"seq":60129542148,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297544,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074296,"statfs":{"total":96636764160,"available":96332079104,"internally_reserved":0,"allocated":188416,"data_stored":62739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0409999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0960000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4750000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2190000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.403}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297548,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074292,"statfs":{"total":96636764160,"available":96332075008,"internally_reserved":0,"allocated":188416,"data_stored":62739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1970000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0980000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0249999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0649999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5980000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.552}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297552,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074288,"statfs":{"total":96636764160,"available":96332070912,"internally_reserved":0,"allocated":192512,"data_stored":62758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93400000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.081}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2649999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2130000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298196,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073644,"statfs":{"total":96636764160,"available":96331411456,"internally_reserved":0,"allocated":720896,"data_stored":591667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0509999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94199999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2909999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.24}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T03:36:46.291 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T03:36:46.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.383+0000 7f704c3e4700 1 Processor -- start 2024-09-15T03:36:46.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.383+0000 7f704c3e4700 1 -- start start 2024-09-15T03:36:46.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.383+0000 7f704c3e4700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f70440aeae0 0x7f70440aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:46.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.383+0000 7f704c3e4700 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70440a6850 con 0x7f70440a4d70 2024-09-15T03:36:46.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.383+0000 7f704c3e4700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70441415c0 con 0x7f704413a620 2024-09-15T03:36:46.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.383+0000 7f704c3e4700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70440af430 con 0x7f70440aeae0 2024-09-15T03:36:46.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.383+0000 7f704997f700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f70440aeae0 0x7f70440aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:46.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.383+0000 7f704997f700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f70440aeae0 0x7f70440aeef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3301/0 says I am v2:172.21.15.110:38882/0 (socket says 172.21.15.110:38882) 2024-09-15T03:36:46.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.383+0000 7f704997f700 1 -- 172.21.15.110:0/3474378466 learned_addr learned my addr 172.21.15.110:0/3474378466 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:36:46.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.383+0000 7f704997f700 1 -- 172.21.15.110:0/3474378466 >> v1:172.21.15.177:6789/0 conn(0x7f704413a620 legacy=0x7f704413aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:36:46.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.383+0000 7f704997f700 1 -- 172.21.15.110:0/3474378466 >> v1:172.21.15.110:6789/0 conn(0x7f70440a4d70 legacy=0x7f7044140d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:46.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.383+0000 7f704997f700 1 -- 172.21.15.110:0/3474378466 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70440a5140 con 0x7f70440aeae0 2024-09-15T03:36:46.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.383+0000 7f704997f700 1 --2- 172.21.15.110:0/3474378466 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f70440aeae0 0x7f70440aeef0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7040009f50 tx=0x7f7040009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64a377f4bda4fd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:46.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.383+0000 7f704917e700 1 -- 172.21.15.110:0/3474378466 <== mon.2 v2:172.21.15.110:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704000e070 con 0x7f70440aeae0 2024-09-15T03:36:46.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.383+0000 7f704917e700 1 -- 172.21.15.110:0/3474378466 <== mon.2 v2:172.21.15.110:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f704000ba70 con 0x7f70440aeae0 2024-09-15T03:36:46.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.383+0000 7f704917e700 1 -- 172.21.15.110:0/3474378466 <== mon.2 v2:172.21.15.110:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70400092e0 con 0x7f70440aeae0 2024-09-15T03:36:46.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.383+0000 7f704c3e4700 1 -- 172.21.15.110:0/3474378466 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f70440aeae0 msgr2=0x7f70440aeef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:46.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.383+0000 7f704c3e4700 1 --2- 172.21.15.110:0/3474378466 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f70440aeae0 0x7f70440aeef0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7040009f50 tx=0x7f7040009840 comp rx=0 tx=0).stop 2024-09-15T03:36:46.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.383+0000 7f704c3e4700 1 -- 172.21.15.110:0/3474378466 shutdown_connections 2024-09-15T03:36:46.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.383+0000 7f704c3e4700 1 --2- 172.21.15.110:0/3474378466 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f70440aeae0 0x7f70440aeef0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:46.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.383+0000 7f704c3e4700 1 -- 172.21.15.110:0/3474378466 >> 172.21.15.110:0/3474378466 conn(0x7f704406f870 msgr2=0x7f70440b4170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:46.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.383+0000 7f704c3e4700 1 -- 172.21.15.110:0/3474378466 shutdown_connections 2024-09-15T03:36:46.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.383+0000 7f704c3e4700 1 -- 172.21.15.110:0/3474378466 wait complete. 2024-09-15T03:36:46.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.387+0000 7f704c3e4700 1 Processor -- start 2024-09-15T03:36:46.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.387+0000 7f704c3e4700 1 -- start start 2024-09-15T03:36:46.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.387+0000 7f704c3e4700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f70440a4d70 0x7f7044150e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:46.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.387+0000 7f704c3e4700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f70440aeae0 0x7f70440a9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:46.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.387+0000 7f704c3e4700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f704413a620 0x7f70440aa220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:46.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.387+0000 7f704c3e4700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70440a9850 con 0x7f70440a4d70 2024-09-15T03:36:46.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.387+0000 7f704c3e4700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70440a8880 con 0x7f70440aeae0 2024-09-15T03:36:46.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.387+0000 7f704c3e4700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70440aa760 con 0x7f704413a620 2024-09-15T03:36:46.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.387+0000 7f704a981700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f704413a620 0x7f70440aa220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:46.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.387+0000 7f704a180700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f70440a4d70 0x7f7044150e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:46.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.387+0000 7f704a981700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f704413a620 0x7f70440aa220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3301/0 says I am v2:172.21.15.110:38890/0 (socket says 172.21.15.110:38890) 2024-09-15T03:36:46.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.387+0000 7f704a981700 1 -- 172.21.15.110:0/348217559 learned_addr learned my addr 172.21.15.110:0/348217559 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:36:46.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.387+0000 7f704997f700 1 --2- 172.21.15.110:0/348217559 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f70440aeae0 0x7f70440a9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:46.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.387+0000 7f704a180700 1 -- 172.21.15.110:0/348217559 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f704413a620 msgr2=0x7f70440aa220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:46.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.387+0000 7f704a180700 1 --2- 172.21.15.110:0/348217559 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f704413a620 0x7f70440aa220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:46.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.387+0000 7f704a180700 1 -- 172.21.15.110:0/348217559 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f70440aeae0 msgr2=0x7f70440a9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:46.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.387+0000 7f704a180700 1 --2- 172.21.15.110:0/348217559 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f70440aeae0 0x7f70440a9ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:46.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.387+0000 7f704a180700 1 -- 172.21.15.110:0/348217559 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7040009ae0 con 0x7f70440a4d70 2024-09-15T03:36:46.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.387+0000 7f704a180700 1 --2- 172.21.15.110:0/348217559 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f70440a4d70 0x7f7044150e70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f703400c490 tx=0x7f7034007ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:46.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.387+0000 7f703a7fc700 1 -- 172.21.15.110:0/348217559 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7034011590 con 0x7f70440a4d70 2024-09-15T03:36:46.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.387+0000 7f703a7fc700 1 -- 172.21.15.110:0/348217559 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7034011710 con 0x7f70440a4d70 2024-09-15T03:36:46.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.387+0000 7f704c3e4700 1 -- 172.21.15.110:0/348217559 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f704414b1e0 con 0x7f70440a4d70 2024-09-15T03:36:46.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.387+0000 7f703a7fc700 1 -- 172.21.15.110:0/348217559 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7034011ad0 con 0x7f70440a4d70 2024-09-15T03:36:46.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.387+0000 7f704c3e4700 1 -- 172.21.15.110:0/348217559 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70441518f0 con 0x7f70440a4d70 2024-09-15T03:36:46.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.391+0000 7f704c3e4700 1 -- 172.21.15.110:0/348217559 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70440a95f0 con 0x7f70440a4d70 2024-09-15T03:36:46.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.391+0000 7f703a7fc700 1 -- 172.21.15.110:0/348217559 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) v1 ==== 85847+0+0 (secure 0 0 0) 0x7f703401b3b0 con 0x7f70440a4d70 2024-09-15T03:36:46.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.391+0000 7f703a7fc700 1 --2- 172.21.15.110:0/348217559 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f7028067a80 0x7f7028069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:46.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.391+0000 7f704997f700 1 --2- 172.21.15.110:0/348217559 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f7028067a80 0x7f7028069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:46.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.391+0000 7f703a7fc700 1 -- 172.21.15.110:0/348217559 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5262+0+0 (secure 0 0 0) 0x7f7034084d20 con 0x7f70440a4d70 2024-09-15T03:36:46.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.391+0000 7f704997f700 1 --2- 172.21.15.110:0/348217559 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f7028067a80 0x7f7028069f30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7040009880 tx=0x7f7040017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:46.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.395+0000 7f703a7fc700 1 -- 172.21.15.110:0/348217559 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f7034051fc0 con 0x7f70440a4d70 2024-09-15T03:36:46.548 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.543+0000 7f704c3e4700 1 -- 172.21.15.110:0/348217559 --> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f70440a9990 con 0x7f7028067a80 2024-09-15T03:36:46.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.547+0000 7f703a7fc700 1 -- 172.21.15.110:0/348217559 <== mgr.4104 v2:172.21.15.110:6832/22190 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48451 (secure 0 0 0) 0x7f704000bd50 con 0x7f7028067a80 2024-09-15T03:36:46.554 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-15T03:36:46.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.551+0000 7f704c3e4700 1 -- 172.21.15.110:0/348217559 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f7028067a80 msgr2=0x7f7028069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:46.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.551+0000 7f704c3e4700 1 --2- 172.21.15.110:0/348217559 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f7028067a80 0x7f7028069f30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7040009880 tx=0x7f7040017000 comp rx=0 tx=0).stop 2024-09-15T03:36:46.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.551+0000 7f704c3e4700 1 -- 172.21.15.110:0/348217559 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f70440a4d70 msgr2=0x7f7044150e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:46.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.551+0000 7f704c3e4700 1 --2- 172.21.15.110:0/348217559 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f70440a4d70 0x7f7044150e70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f703400c490 tx=0x7f7034007ec0 comp rx=0 tx=0).stop 2024-09-15T03:36:46.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.551+0000 7f704c3e4700 1 -- 172.21.15.110:0/348217559 shutdown_connections 2024-09-15T03:36:46.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.551+0000 7f704c3e4700 1 --2- 172.21.15.110:0/348217559 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f7028067a80 0x7f7028069f30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:46.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.551+0000 7f704c3e4700 1 --2- 172.21.15.110:0/348217559 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f704413a620 0x7f70440aa220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:46.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.551+0000 7f704c3e4700 1 --2- 172.21.15.110:0/348217559 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f70440aeae0 0x7f70440a9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:46.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.551+0000 7f704c3e4700 1 --2- 172.21.15.110:0/348217559 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f70440a4d70 0x7f7044150e70 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:46.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.551+0000 7f704c3e4700 1 -- 172.21.15.110:0/348217559 >> 172.21.15.110:0/348217559 conn(0x7f704406f870 msgr2=0x7f7044140b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:46.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.551+0000 7f704c3e4700 1 -- 172.21.15.110:0/348217559 shutdown_connections 2024-09-15T03:36:46.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.551+0000 7f704c3e4700 1 -- 172.21.15.110:0/348217559 wait complete. 2024-09-15T03:36:46.557 INFO:teuthology.orchestra.run.smithi110.stderr:dumped all 2024-09-15T03:36:46.567 INFO:teuthology.orchestra.run.smithi110.stdout:{"pg_ready":true,"pg_map":{"version":16,"stamp":"2024-09-15T03:36:44.604064+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2381552,"kb_used_data":2520,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":752593168,"statfs":{"total":773094113280,"available":770655404032,"internally_reserved":0,"allocated":2580480,"data_stored":1559806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-15T03:36:42.814243+0000","last_change":"2024-09-15T03:36:42.814645+0000","last_active":"2024-09-15T03:36:42.814243+0000","last_peered":"2024-09-15T03:36:42.814243+0000","last_clean":"2024-09-15T03:36:42.814243+0000","last_became_active":"2024-09-15T03:36:40.672828+0000","last_became_peered":"2024-09-15T03:36:40.672828+0000","last_unstale":"2024-09-15T03:36:42.814243+0000","last_undegraded":"2024-09-15T03:36:42.814243+0000","last_fullsized":"2024-09-15T03:36:42.814243+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_clean_scrub_stamp":"2024-09-15T03:36:39.639431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T13:40:53.039569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064069099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-15T03:36:42.686802+0000","last_change":"2024-09-15T03:36:42.687267+0000","last_active":"2024-09-15T03:36:42.686802+0000","last_peered":"2024-09-15T03:36:42.686802+0000","last_clean":"2024-09-15T03:36:42.686802+0000","last_became_active":"2024-09-15T03:36:40.671677+0000","last_became_peered":"2024-09-15T03:36:40.671677+0000","last_unstale":"2024-09-15T03:36:42.686802+0000","last_undegraded":"2024-09-15T03:36:42.686802+0000","last_fullsized":"2024-09-15T03:36:42.686802+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_clean_scrub_stamp":"2024-09-15T03:36:39.639431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:55:15.493964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000861089,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-15T03:36:42.853958+0000","last_change":"2024-09-15T03:36:42.854270+0000","last_active":"2024-09-15T03:36:42.853958+0000","last_peered":"2024-09-15T03:36:42.853958+0000","last_clean":"2024-09-15T03:36:42.853958+0000","last_became_active":"2024-09-15T03:36:41.079216+0000","last_became_peered":"2024-09-15T03:36:41.079216+0000","last_unstale":"2024-09-15T03:36:42.853958+0000","last_undegraded":"2024-09-15T03:36:42.853958+0000","last_fullsized":"2024-09-15T03:36:42.853958+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_clean_scrub_stamp":"2024-09-15T03:36:39.639431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:19:37.610607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062568200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-15T03:36:42.686886+0000","last_change":"2024-09-15T03:36:42.687270+0000","last_active":"2024-09-15T03:36:42.686886+0000","last_peered":"2024-09-15T03:36:42.686886+0000","last_clean":"2024-09-15T03:36:42.686886+0000","last_became_active":"2024-09-15T03:36:41.077266+0000","last_became_peered":"2024-09-15T03:36:41.077266+0000","last_unstale":"2024-09-15T03:36:42.686886+0000","last_undegraded":"2024-09-15T03:36:42.686886+0000","last_fullsized":"2024-09-15T03:36:42.686886+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_clean_scrub_stamp":"2024-09-15T03:36:39.639431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:54:58.435053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067514400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"19'2","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-15T03:36:42.695885+0000","last_change":"2024-09-15T03:36:42.695885+0000","last_active":"2024-09-15T03:36:42.695885+0000","last_peered":"2024-09-15T03:36:42.695885+0000","last_clean":"2024-09-15T03:36:42.695885+0000","last_became_active":"2024-09-15T03:36:40.673743+0000","last_became_peered":"2024-09-15T03:36:40.673743+0000","last_unstale":"2024-09-15T03:36:42.695885+0000","last_undegraded":"2024-09-15T03:36:42.695885+0000","last_fullsized":"2024-09-15T03:36:42.695885+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_clean_scrub_stamp":"2024-09-15T03:36:39.639431+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T13:57:31.462981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002159518,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-15T03:36:42.801323+0000","last_change":"2024-09-15T03:36:42.801569+0000","last_active":"2024-09-15T03:36:42.801323+0000","last_peered":"2024-09-15T03:36:42.801323+0000","last_clean":"2024-09-15T03:36:42.801323+0000","last_became_active":"2024-09-15T03:36:40.669392+0000","last_became_peered":"2024-09-15T03:36:40.669392+0000","last_unstale":"2024-09-15T03:36:42.801323+0000","last_undegraded":"2024-09-15T03:36:42.801323+0000","last_fullsized":"2024-09-15T03:36:42.801323+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_clean_scrub_stamp":"2024-09-15T03:36:39.639431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:00:33.102146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059629799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-15T03:36:42.853828+0000","last_change":"2024-09-15T03:36:42.854222+0000","last_active":"2024-09-15T03:36:42.853828+0000","last_peered":"2024-09-15T03:36:42.853828+0000","last_clean":"2024-09-15T03:36:42.853828+0000","last_became_active":"2024-09-15T03:36:40.672416+0000","last_became_peered":"2024-09-15T03:36:40.672416+0000","last_unstale":"2024-09-15T03:36:42.853828+0000","last_undegraded":"2024-09-15T03:36:42.853828+0000","last_fullsized":"2024-09-15T03:36:42.853828+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_clean_scrub_stamp":"2024-09-15T03:36:39.639431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T15:08:21.420921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065187600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"17'1","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-15T03:36:42.694075+0000","last_change":"2024-09-15T03:36:42.694486+0000","last_active":"2024-09-15T03:36:42.694075+0000","last_peered":"2024-09-15T03:36:42.694075+0000","last_clean":"2024-09-15T03:36:42.694075+0000","last_became_active":"2024-09-15T03:36:40.674442+0000","last_became_peered":"2024-09-15T03:36:40.674442+0000","last_unstale":"2024-09-15T03:36:42.694075+0000","last_undegraded":"2024-09-15T03:36:42.694075+0000","last_fullsized":"2024-09-15T03:36:42.694075+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_clean_scrub_stamp":"2024-09-15T03:36:39.639431+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:25:57.474430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067001500000000004,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"16'72","reported_seq":102,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T03:36:42.541403+0000","last_change":"2024-09-15T03:36:38.815652+0000","last_active":"2024-09-15T03:36:42.541403+0000","last_peered":"2024-09-15T03:36:42.541403+0000","last_clean":"2024-09-15T03:36:42.541403+0000","last_became_active":"2024-09-15T03:36:38.813490+0000","last_became_peered":"2024-09-15T03:36:38.813490+0000","last_unstale":"2024-09-15T03:36:42.541403+0000","last_undegraded":"2024-09-15T03:36:42.541403+0000","last_fullsized":"2024-09-15T03:36:42.541403+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:37.618040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:37.618040+0000","last_clean_scrub_stamp":"2024-09-15T03:36:37.618040+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:02:12.095255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542148,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298068,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073772,"statfs":{"total":96636764160,"available":96331542528,"internally_reserved":0,"allocated":720896,"data_stored":591667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97099999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4330000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5089999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.645}]}]},{"osd":6,"up_from":14,"seq":60129542147,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297548,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074292,"statfs":{"total":96636764160,"available":96332075008,"internally_reserved":0,"allocated":188416,"data_stored":62739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2509999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.476}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7210000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.095}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.873}]}]},{"osd":5,"up_from":14,"seq":60129542147,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297548,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074292,"statfs":{"total":96636764160,"available":96332075008,"internally_reserved":0,"allocated":192512,"data_stored":62758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92700000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.048}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.145}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5660000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.484}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.171}]}]},{"osd":3,"up_from":14,"seq":60129542147,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297548,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074292,"statfs":{"total":96636764160,"available":96332075008,"internally_reserved":0,"allocated":188416,"data_stored":62739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.05}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.085}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.157}]}]},{"osd":2,"up_from":14,"seq":60129542148,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297544,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074296,"statfs":{"total":96636764160,"available":96332079104,"internally_reserved":0,"allocated":188416,"data_stored":62739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0409999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0960000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4750000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2190000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.403}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297548,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074292,"statfs":{"total":96636764160,"available":96332075008,"internally_reserved":0,"allocated":188416,"data_stored":62739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1970000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0980000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0249999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0649999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5980000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.552}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297552,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074288,"statfs":{"total":96636764160,"available":96332070912,"internally_reserved":0,"allocated":192512,"data_stored":62758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93400000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.081}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2649999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2130000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298196,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073644,"statfs":{"total":96636764160,"available":96331411456,"internally_reserved":0,"allocated":720896,"data_stored":591667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0509999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94199999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2909999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.24}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T03:36:46.568 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-15T03:36:46.568 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-15T03:36:46.568 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-15T03:36:46.568 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-15T03:36:46.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.655+0000 7fb4c28c9700 1 Processor -- start 2024-09-15T03:36:46.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.655+0000 7fb4c28c9700 1 -- start start 2024-09-15T03:36:46.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.655+0000 7fb4c28c9700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fb4bc068680 0x7fb4bc140d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:46.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.655+0000 7fb4c28c9700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fb4bc0a9300 0x7fb4bc13b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:46.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.655+0000 7fb4c28c9700 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4bc0aa2f0 con 0x7fb4bc068a50 2024-09-15T03:36:46.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.655+0000 7fb4c28c9700 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4bc0a8e00 con 0x7fb4bc0a9300 2024-09-15T03:36:46.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.655+0000 7fb4c28c9700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4bc0acc60 con 0x7fb4bc068680 2024-09-15T03:36:46.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.655+0000 7fb4c0e66700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fb4bc0a9300 0x7fb4bc13b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:46.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.655+0000 7fb4c0e66700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fb4bc0a9300 0x7fb4bc13b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44372/0 (socket says 172.21.15.110:44372) 2024-09-15T03:36:46.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.655+0000 7fb4c0e66700 1 -- 172.21.15.110:0/4067699817 learned_addr learned my addr 172.21.15.110:0/4067699817 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:36:46.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.655+0000 7fb4bbfff700 1 --2- 172.21.15.110:0/4067699817 >> v2:172.21.15.177:3300/0 conn(0x7fb4bc068680 0x7fb4bc140d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:46.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.659+0000 7fb4c0e66700 1 -- 172.21.15.110:0/4067699817 >> v2:172.21.15.177:3300/0 conn(0x7fb4bc068680 msgr2=0x7fb4bc140d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:36:46.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.659+0000 7fb4c0e66700 1 --2- 172.21.15.110:0/4067699817 >> v2:172.21.15.177:3300/0 conn(0x7fb4bc068680 0x7fb4bc140d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:46.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.659+0000 7fb4c0e66700 1 -- 172.21.15.110:0/4067699817 >> v1:172.21.15.110:6789/0 conn(0x7fb4bc068a50 legacy=0x7fb4bc13a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:46.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.659+0000 7fb4c0e66700 1 -- 172.21.15.110:0/4067699817 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4bc0a88e0 con 0x7fb4bc0a9300 2024-09-15T03:36:46.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.659+0000 7fb4c0e66700 1 --2- 172.21.15.110:0/4067699817 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4bc0a9300 0x7fb4bc13b030 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc06bff0 tx=0x7fb4b4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32331987a8d66e75 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:46.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.659+0000 7fb4baffd700 1 -- 172.21.15.110:0/4067699817 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b400e070 con 0x7fb4bc0a9300 2024-09-15T03:36:46.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.659+0000 7fb4baffd700 1 -- 172.21.15.110:0/4067699817 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4b400ba50 con 0x7fb4bc0a9300 2024-09-15T03:36:46.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.659+0000 7fb4baffd700 1 -- 172.21.15.110:0/4067699817 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b40092e0 con 0x7fb4bc0a9300 2024-09-15T03:36:46.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.659+0000 7fb4c28c9700 1 -- 172.21.15.110:0/4067699817 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4bc0a9300 msgr2=0x7fb4bc13b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:46.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.659+0000 7fb4c28c9700 1 --2- 172.21.15.110:0/4067699817 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4bc0a9300 0x7fb4bc13b030 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc06bff0 tx=0x7fb4b4009840 comp rx=0 tx=0).stop 2024-09-15T03:36:46.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.659+0000 7fb4c28c9700 1 -- 172.21.15.110:0/4067699817 shutdown_connections 2024-09-15T03:36:46.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.659+0000 7fb4c28c9700 1 --2- 172.21.15.110:0/4067699817 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4bc0a9300 0x7fb4bc13b030 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:46.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.659+0000 7fb4c28c9700 1 --2- 172.21.15.110:0/4067699817 >> v2:172.21.15.177:3300/0 conn(0x7fb4bc068680 0x7fb4bc140d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:46.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.659+0000 7fb4c28c9700 1 -- 172.21.15.110:0/4067699817 >> 172.21.15.110:0/4067699817 conn(0x7fb4bc0ad3d0 msgr2=0x7fb4bc0ad7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:46.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.659+0000 7fb4c28c9700 1 -- 172.21.15.110:0/4067699817 shutdown_connections 2024-09-15T03:36:46.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.659+0000 7fb4c28c9700 1 -- 172.21.15.110:0/4067699817 wait complete. 2024-09-15T03:36:46.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.663+0000 7fb4c28c9700 1 Processor -- start 2024-09-15T03:36:46.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.663+0000 7fb4c28c9700 1 -- start start 2024-09-15T03:36:46.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.663+0000 7fb4c28c9700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4bc068680 0x7fb4bc06e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:46.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.663+0000 7fb4c28c9700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4bc068a50 0x7fb4bc13b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:46.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.663+0000 7fb4c28c9700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fb4bc0a9300 0x7fb4bc150840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:46.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.663+0000 7fb4c28c9700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4bc1191d0 con 0x7fb4bc068a50 2024-09-15T03:36:46.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.663+0000 7fb4c28c9700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4bc13dc20 con 0x7fb4bc068680 2024-09-15T03:36:46.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.663+0000 7fb4c28c9700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4bc13d950 con 0x7fb4bc0a9300 2024-09-15T03:36:46.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.663+0000 7fb4c0e66700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fb4bc0a9300 0x7fb4bc150840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:46.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.663+0000 7fb4c0e66700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fb4bc0a9300 0x7fb4bc150840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3301/0 says I am v2:172.21.15.110:38898/0 (socket says 172.21.15.110:38898) 2024-09-15T03:36:46.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.663+0000 7fb4c0e66700 1 -- 172.21.15.110:0/3757449633 learned_addr learned my addr 172.21.15.110:0/3757449633 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:36:46.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.663+0000 7fb4bb7fe700 1 --2- 172.21.15.110:0/3757449633 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4bc068a50 0x7fb4bc13b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:46.670 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.663+0000 7fb4bbfff700 1 --2- 172.21.15.110:0/3757449633 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4bc068680 0x7fb4bc06e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:46.670 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.663+0000 7fb4bb7fe700 1 -- 172.21.15.110:0/3757449633 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fb4bc0a9300 msgr2=0x7fb4bc150840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:46.670 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.663+0000 7fb4bb7fe700 1 --2- 172.21.15.110:0/3757449633 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fb4bc0a9300 0x7fb4bc150840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:46.670 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.663+0000 7fb4bb7fe700 1 -- 172.21.15.110:0/3757449633 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4bc068680 msgr2=0x7fb4bc06e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:46.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.663+0000 7fb4bb7fe700 1 --2- 172.21.15.110:0/3757449633 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4bc068680 0x7fb4bc06e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:46.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.663+0000 7fb4bb7fe700 1 -- 172.21.15.110:0/3757449633 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4b4009ae0 con 0x7fb4bc068a50 2024-09-15T03:36:46.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.663+0000 7fb4bb7fe700 1 --2- 172.21.15.110:0/3757449633 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4bc068a50 0x7fb4bc13b590 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc0aa220 tx=0x7fb4a4007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:46.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.663+0000 7fb4b3fff700 1 -- 172.21.15.110:0/3757449633 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a4011590 con 0x7fb4bc068a50 2024-09-15T03:36:46.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.663+0000 7fb4b3fff700 1 -- 172.21.15.110:0/3757449633 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4a4011710 con 0x7fb4bc068a50 2024-09-15T03:36:46.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.663+0000 7fb4c28c9700 1 -- 172.21.15.110:0/3757449633 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4bc06f110 con 0x7fb4bc068a50 2024-09-15T03:36:46.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.663+0000 7fb4b3fff700 1 -- 172.21.15.110:0/3757449633 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a4011ad0 con 0x7fb4bc068a50 2024-09-15T03:36:46.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.663+0000 7fb4c28c9700 1 -- 172.21.15.110:0/3757449633 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4bc13d500 con 0x7fb4bc068a50 2024-09-15T03:36:46.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.667+0000 7fb4b3fff700 1 -- 172.21.15.110:0/3757449633 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) v1 ==== 85847+0+0 (secure 0 0 0) 0x7fb4a401c460 con 0x7fb4bc068a50 2024-09-15T03:36:46.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.667+0000 7fb4c28c9700 1 -- 172.21.15.110:0/3757449633 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4bc0716d0 con 0x7fb4bc068a50 2024-09-15T03:36:46.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.667+0000 7fb4b3fff700 1 --2- 172.21.15.110:0/3757449633 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fb49c067a80 0x7fb49c069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:46.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.667+0000 7fb4b3fff700 1 -- 172.21.15.110:0/3757449633 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5262+0+0 (secure 0 0 0) 0x7fb4a4084db0 con 0x7fb4bc068a50 2024-09-15T03:36:46.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.667+0000 7fb4bbfff700 1 --2- 172.21.15.110:0/3757449633 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fb49c067a80 0x7fb49c069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:46.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.667+0000 7fb4bbfff700 1 --2- 172.21.15.110:0/3757449633 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fb49c067a80 0x7fb49c069f30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac006fd0 tx=0x7fb4ac008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:46.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.675+0000 7fb4b3fff700 1 -- 172.21.15.110:0/3757449633 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fb4a4052070 con 0x7fb4bc068a50 2024-09-15T03:36:46.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.859+0000 7fb4c28c9700 1 -- 172.21.15.110:0/3757449633 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fb4bc06f110 con 0x7fb4bc068a50 2024-09-15T03:36:46.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.859+0000 7fb4b3fff700 1 -- 172.21.15.110:0/3757449633 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fb4bc06f110 con 0x7fb4bc068a50 2024-09-15T03:36:46.864 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-15T03:36:46.865 INFO:teuthology.orchestra.run.smithi110.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-15T03:36:46.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.859+0000 7fb4c28c9700 1 -- 172.21.15.110:0/3757449633 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fb49c067a80 msgr2=0x7fb49c069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:46.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.859+0000 7fb4c28c9700 1 --2- 172.21.15.110:0/3757449633 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fb49c067a80 0x7fb49c069f30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac006fd0 tx=0x7fb4ac008040 comp rx=0 tx=0).stop 2024-09-15T03:36:46.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.859+0000 7fb4c28c9700 1 -- 172.21.15.110:0/3757449633 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4bc068a50 msgr2=0x7fb4bc13b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:46.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.859+0000 7fb4c28c9700 1 --2- 172.21.15.110:0/3757449633 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4bc068a50 0x7fb4bc13b590 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc0aa220 tx=0x7fb4a4007ea0 comp rx=0 tx=0).stop 2024-09-15T03:36:46.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.859+0000 7fb4c28c9700 1 -- 172.21.15.110:0/3757449633 shutdown_connections 2024-09-15T03:36:46.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.859+0000 7fb4c28c9700 1 --2- 172.21.15.110:0/3757449633 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fb49c067a80 0x7fb49c069f30 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:46.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.859+0000 7fb4c28c9700 1 --2- 172.21.15.110:0/3757449633 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fb4bc0a9300 0x7fb4bc150840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:46.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.859+0000 7fb4c28c9700 1 --2- 172.21.15.110:0/3757449633 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4bc068a50 0x7fb4bc13b590 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:46.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.859+0000 7fb4c28c9700 1 --2- 172.21.15.110:0/3757449633 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4bc068680 0x7fb4bc06e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:46.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.859+0000 7fb4c28c9700 1 -- 172.21.15.110:0/3757449633 >> 172.21.15.110:0/3757449633 conn(0x7fb4bc0ad3d0 msgr2=0x7fb4bc0ad7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:46.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.863+0000 7fb4c28c9700 1 -- 172.21.15.110:0/3757449633 shutdown_connections 2024-09-15T03:36:46.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.863+0000 7fb4c28c9700 1 -- 172.21.15.110:0/3757449633 wait complete. 2024-09-15T03:36:46.883 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-15T03:36:46.884 INFO:teuthology.run_tasks:Running task exec... 2024-09-15T03:36:46.894 INFO:teuthology.task.exec:Executing custom commands... 2024-09-15T03:36:46.895 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi110.front.sepia.ceph.com 2024-09-15T03:36:46.895 DEBUG:teuthology.orchestra.run.smithi110:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph osd pool create cephsqlite' 2024-09-15T03:36:46.998 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.991+0000 7f5e2641b700 1 Processor -- start 2024-09-15T03:36:46.998 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.991+0000 7f5e2641b700 1 -- start start 2024-09-15T03:36:46.998 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.991+0000 7f5e2641b700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f5e200ad2e0 0x7f5e2013a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:46.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.991+0000 7f5e2641b700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f5e2013ad80 0x7f5e2006c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:46.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.991+0000 7f5e2641b700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f5e2006c920 0x7f5e2013dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:46.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.991+0000 7f5e2641b700 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e200ad6b0 con 0x7f5e2006c920 2024-09-15T03:36:46.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.991+0000 7f5e2641b700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e2006bd60 con 0x7f5e200ad2e0 2024-09-15T03:36:46.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.991+0000 7f5e2641b700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2006db70 con 0x7f5e2013ad80 2024-09-15T03:36:46.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.991+0000 7f5e24c18700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f5e2013ad80 0x7f5e2006c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:46.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.991+0000 7f5e25c1a700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f5e2006c920 0x7f5e2013dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:46.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.991+0000 7f5e24c18700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f5e2013ad80 0x7f5e2006c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3301/0 says I am v2:172.21.15.110:38904/0 (socket says 172.21.15.110:38904) 2024-09-15T03:36:46.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.991+0000 7f5e24c18700 1 -- 172.21.15.110:0/1618626326 learned_addr learned my addr 172.21.15.110:0/1618626326 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:36:47.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.991+0000 7f5e25419700 1 --2- 172.21.15.110:0/1618626326 >> v2:172.21.15.177:3300/0 conn(0x7f5e200ad2e0 0x7f5e2013a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:47.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.991+0000 7f5e25c1a700 1 -- 172.21.15.110:0/1618626326 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f5e2013ad80 msgr2=0x7f5e2006c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:36:47.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.991+0000 7f5e25c1a700 1 --2- 172.21.15.110:0/1618626326 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f5e2013ad80 0x7f5e2006c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:47.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.991+0000 7f5e25c1a700 1 -- 172.21.15.110:0/1618626326 >> v2:172.21.15.177:3300/0 conn(0x7f5e200ad2e0 msgr2=0x7f5e2013a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:36:47.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.991+0000 7f5e25c1a700 1 --2- 172.21.15.110:0/1618626326 >> v2:172.21.15.177:3300/0 conn(0x7f5e200ad2e0 0x7f5e2013a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:47.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.991+0000 7f5e25c1a700 1 -- 172.21.15.110:0/1618626326 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e2006cda0 con 0x7f5e2006c920 2024-09-15T03:36:47.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.991+0000 7f5e25419700 1 --2- 172.21.15.110:0/1618626326 >> v2:172.21.15.177:3300/0 conn(0x7f5e200ad2e0 0x7f5e2013a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T03:36:47.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.991+0000 7f5e24c18700 1 --2- 172.21.15.110:0/1618626326 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f5e2013ad80 0x7f5e2006c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T03:36:47.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.991+0000 7f5e25c1a700 1 --2- 172.21.15.110:0/1618626326 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5e2006c920 0x7f5e2013dc70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5e20071840 tx=0x7f5e18009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8762bcd7292ef190 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:47.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.995+0000 7f5e17fff700 1 -- 172.21.15.110:0/1618626326 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1800e070 con 0x7f5e2006c920 2024-09-15T03:36:47.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.995+0000 7f5e17fff700 1 -- 172.21.15.110:0/1618626326 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e1800bb60 con 0x7f5e2006c920 2024-09-15T03:36:47.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.995+0000 7f5e17fff700 1 -- 172.21.15.110:0/1618626326 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e18012430 con 0x7f5e2006c920 2024-09-15T03:36:47.002 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.995+0000 7f5e2641b700 1 -- 172.21.15.110:0/1618626326 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5e2006c920 msgr2=0x7f5e2013dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:47.002 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.995+0000 7f5e2641b700 1 --2- 172.21.15.110:0/1618626326 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5e2006c920 0x7f5e2013dc70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5e20071840 tx=0x7f5e18009780 comp rx=0 tx=0).stop 2024-09-15T03:36:47.002 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.995+0000 7f5e2641b700 1 -- 172.21.15.110:0/1618626326 shutdown_connections 2024-09-15T03:36:47.002 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.995+0000 7f5e2641b700 1 --2- 172.21.15.110:0/1618626326 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5e2006c920 0x7f5e2013dc70 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:47.002 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.995+0000 7f5e2641b700 1 --2- 172.21.15.110:0/1618626326 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f5e2013ad80 0x7f5e2006c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:47.002 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.995+0000 7f5e2641b700 1 --2- 172.21.15.110:0/1618626326 >> v2:172.21.15.177:3300/0 conn(0x7f5e200ad2e0 0x7f5e2013a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:47.002 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.995+0000 7f5e2641b700 1 -- 172.21.15.110:0/1618626326 >> 172.21.15.110:0/1618626326 conn(0x7f5e200b0970 msgr2=0x7f5e200ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:47.002 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.995+0000 7f5e2641b700 1 -- 172.21.15.110:0/1618626326 shutdown_connections 2024-09-15T03:36:47.002 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.995+0000 7f5e2641b700 1 -- 172.21.15.110:0/1618626326 wait complete. 2024-09-15T03:36:47.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.995+0000 7f5e2641b700 1 Processor -- start 2024-09-15T03:36:47.003 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.995+0000 7f5e2641b700 1 -- start start 2024-09-15T03:36:47.005 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.999+0000 7f5e2641b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5e2006c920 0x7f5e200a9b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:47.005 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.999+0000 7f5e2641b700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f5e200ad2e0 0x7f5e201500e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:47.005 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.999+0000 7f5e2641b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e2013ad80 0x7f5e20150620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:47.005 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.999+0000 7f5e2641b700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2013eb80 con 0x7f5e2006c920 2024-09-15T03:36:47.005 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.999+0000 7f5e2641b700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2013fca0 con 0x7f5e2013ad80 2024-09-15T03:36:47.005 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.999+0000 7f5e2641b700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2013ff10 con 0x7f5e200ad2e0 2024-09-15T03:36:47.005 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.999+0000 7f5e25419700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5e2006c920 0x7f5e200a9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:47.005 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.999+0000 7f5e25419700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5e2006c920 0x7f5e200a9b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44398/0 (socket says 172.21.15.110:44398) 2024-09-15T03:36:47.005 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.999+0000 7f5e25419700 1 -- 172.21.15.110:0/385806460 learned_addr learned my addr 172.21.15.110:0/385806460 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:36:47.005 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.999+0000 7f5e24c18700 1 --2- 172.21.15.110:0/385806460 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f5e200ad2e0 0x7f5e201500e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:47.006 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.999+0000 7f5e25c1a700 1 --2- 172.21.15.110:0/385806460 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e2013ad80 0x7f5e20150620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:47.006 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.999+0000 7f5e25419700 1 -- 172.21.15.110:0/385806460 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f5e200ad2e0 msgr2=0x7f5e201500e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:47.006 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.999+0000 7f5e25419700 1 --2- 172.21.15.110:0/385806460 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f5e200ad2e0 0x7f5e201500e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:47.006 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.999+0000 7f5e25419700 1 -- 172.21.15.110:0/385806460 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e2013ad80 msgr2=0x7f5e20150620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:47.006 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.999+0000 7f5e25419700 1 --2- 172.21.15.110:0/385806460 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e2013ad80 0x7f5e20150620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:47.006 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.999+0000 7f5e25419700 1 -- 172.21.15.110:0/385806460 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e100094f0 con 0x7f5e2006c920 2024-09-15T03:36:47.006 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.999+0000 7f5e25c1a700 1 --2- 172.21.15.110:0/385806460 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e2013ad80 0x7f5e20150620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T03:36:47.006 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.999+0000 7f5e25419700 1 --2- 172.21.15.110:0/385806460 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5e2006c920 0x7f5e200a9b50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c00e900 tx=0x7f5e1c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:47.006 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.999+0000 7f5e157fa700 1 -- 172.21.15.110:0/385806460 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c013410 con 0x7f5e2006c920 2024-09-15T03:36:47.006 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.999+0000 7f5e157fa700 1 -- 172.21.15.110:0/385806460 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e1c013570 con 0x7f5e2006c920 2024-09-15T03:36:47.006 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.999+0000 7f5e2641b700 1 -- 172.21.15.110:0/385806460 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e18009a20 con 0x7f5e2006c920 2024-09-15T03:36:47.006 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.999+0000 7f5e157fa700 1 -- 172.21.15.110:0/385806460 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c013910 con 0x7f5e2006c920 2024-09-15T03:36:47.007 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.999+0000 7f5e2641b700 1 -- 172.21.15.110:0/385806460 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e2013d790 con 0x7f5e2006c920 2024-09-15T03:36:47.007 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.999+0000 7f5e2641b700 1 -- 172.21.15.110:0/385806460 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e20066dc0 con 0x7f5e2006c920 2024-09-15T03:36:47.007 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:46.999+0000 7f5e157fa700 1 -- 172.21.15.110:0/385806460 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) v1 ==== 85847+0+0 (secure 0 0 0) 0x7f5e1c013d00 con 0x7f5e2006c920 2024-09-15T03:36:47.007 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.003+0000 7f5e157fa700 1 --2- 172.21.15.110:0/385806460 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f5e04067a80 0x7f5e04069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:47.007 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.003+0000 7f5e157fa700 1 -- 172.21.15.110:0/385806460 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5262+0+0 (secure 0 0 0) 0x7f5e1c085330 con 0x7f5e2006c920 2024-09-15T03:36:47.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.003+0000 7f5e24c18700 1 --2- 172.21.15.110:0/385806460 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f5e04067a80 0x7f5e04069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:47.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.003+0000 7f5e24c18700 1 --2- 172.21.15.110:0/385806460 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f5e04067a80 0x7f5e04069f30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5e200aa490 tx=0x7f5e10009250 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:47.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.007+0000 7f5e157fa700 1 -- 172.21.15.110:0/385806460 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f5e1c051e60 con 0x7f5e2006c920 2024-09-15T03:36:47.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.159+0000 7f5e2641b700 1 -- 172.21.15.110:0/385806460 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephsqlite"} v 0) v1 -- 0x7f5e2013dd70 con 0x7f5e2006c920 2024-09-15T03:36:47.725 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.719+0000 7f5e157fa700 1 -- 172.21.15.110:0/385806460 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephsqlite"}]=0 pool 'cephsqlite' created v20) v1 ==== 110+0+0 (secure 0 0 0) 0x7f5e1c055670 con 0x7f5e2006c920 2024-09-15T03:36:47.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.727+0000 7f5e2641b700 1 -- 172.21.15.110:0/385806460 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f5e04067a80 msgr2=0x7f5e04069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:47.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.727+0000 7f5e2641b700 1 --2- 172.21.15.110:0/385806460 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f5e04067a80 0x7f5e04069f30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5e200aa490 tx=0x7f5e10009250 comp rx=0 tx=0).stop 2024-09-15T03:36:47.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.727+0000 7f5e2641b700 1 -- 172.21.15.110:0/385806460 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5e2006c920 msgr2=0x7f5e200a9b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:47.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.727+0000 7f5e2641b700 1 --2- 172.21.15.110:0/385806460 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5e2006c920 0x7f5e200a9b50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c00e900 tx=0x7f5e1c008040 comp rx=0 tx=0).stop 2024-09-15T03:36:47.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.727+0000 7f5e2641b700 1 -- 172.21.15.110:0/385806460 shutdown_connections 2024-09-15T03:36:47.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.727+0000 7f5e2641b700 1 --2- 172.21.15.110:0/385806460 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f5e04067a80 0x7f5e04069f30 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:47.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.727+0000 7f5e2641b700 1 --2- 172.21.15.110:0/385806460 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e2013ad80 0x7f5e20150620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:47.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.727+0000 7f5e2641b700 1 --2- 172.21.15.110:0/385806460 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f5e200ad2e0 0x7f5e201500e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:47.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.727+0000 7f5e2641b700 1 --2- 172.21.15.110:0/385806460 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5e2006c920 0x7f5e200a9b50 secure :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c00e900 tx=0x7f5e1c008040 comp rx=0 tx=0).stop 2024-09-15T03:36:47.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.727+0000 7f5e2641b700 1 -- 172.21.15.110:0/385806460 >> 172.21.15.110:0/385806460 conn(0x7f5e200b0970 msgr2=0x7f5e2006b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:47.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.727+0000 7f5e2641b700 1 -- 172.21.15.110:0/385806460 shutdown_connections 2024-09-15T03:36:47.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.727+0000 7f5e2641b700 1 -- 172.21.15.110:0/385806460 wait complete. 2024-09-15T03:36:47.734 INFO:teuthology.orchestra.run.smithi110.stderr:pool 'cephsqlite' created 2024-09-15T03:36:47.744 DEBUG:teuthology.orchestra.run.smithi110:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph auth get-or-create client.libcephsqlite mon '"'"'profile simple-rados-client-with-blocklist'"'"' osd '"'"'allow rwx pool=cephsqlite'"'"' >> /etc/ceph/ceph.keyring' 2024-09-15T03:36:47.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.847+0000 7effa0cb5700 1 Processor -- start 2024-09-15T03:36:47.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.847+0000 7effa0cb5700 1 -- start start 2024-09-15T03:36:47.851 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.847+0000 7effa0cb5700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7eff9c0af4a0 0x7eff9c1411c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:47.851 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.847+0000 7effa0cb5700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7eff9c141700 0x7eff9c13da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:47.851 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.847+0000 7effa0cb5700 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff9c140bd0 con 0x7eff9c0af0d0 2024-09-15T03:36:47.852 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.847+0000 7effa0cb5700 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff9c06b7f0 con 0x7eff9c0af4a0 2024-09-15T03:36:47.852 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.847+0000 7effa0cb5700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff9c0ae8b0 con 0x7eff9c141700 2024-09-15T03:36:47.852 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.847+0000 7eff9bfff700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7eff9c141700 0x7eff9c13da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:47.852 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.847+0000 7eff9b7fe700 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7eff9c0af0d0 0x7eff9c06a580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:43782/0 (socket says 172.21.15.110:43782) 2024-09-15T03:36:47.852 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.847+0000 7eff9b7fe700 1 -- 172.21.15.110:0/1762529320 learned_addr learned my addr 172.21.15.110:0/1762529320 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-15T03:36:47.852 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.847+0000 7eff9affd700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7eff9c0af4a0 0x7eff9c1411c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:47.853 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.847+0000 7eff9bfff700 1 -- 172.21.15.110:0/1762529320 >> v2:172.21.15.110:3300/0 conn(0x7eff9c0af4a0 msgr2=0x7eff9c1411c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:36:47.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.847+0000 7eff9bfff700 1 --2- 172.21.15.110:0/1762529320 >> v2:172.21.15.110:3300/0 conn(0x7eff9c0af4a0 0x7eff9c1411c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:47.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.847+0000 7eff9bfff700 1 -- 172.21.15.110:0/1762529320 >> v1:172.21.15.110:6789/0 conn(0x7eff9c0af0d0 legacy=0x7eff9c06a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:47.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.847+0000 7eff9bfff700 1 -- 172.21.15.110:0/1762529320 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff9c13df80 con 0x7eff9c141700 2024-09-15T03:36:47.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.847+0000 7eff9bfff700 1 --2- 172.21.15.110:0/1762529320 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7eff9c141700 0x7eff9c13da40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7eff94009e60 tx=0x7eff94009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4795547c9b677183 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:47.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.847+0000 7eff9a7fc700 1 -- 172.21.15.110:0/1762529320 <== mon.2 v2:172.21.15.110:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9400e070 con 0x7eff9c141700 2024-09-15T03:36:47.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.847+0000 7eff9a7fc700 1 -- 172.21.15.110:0/1762529320 <== mon.2 v2:172.21.15.110:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff9400bb60 con 0x7eff9c141700 2024-09-15T03:36:47.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.847+0000 7eff9a7fc700 1 -- 172.21.15.110:0/1762529320 <== mon.2 v2:172.21.15.110:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff94012430 con 0x7eff9c141700 2024-09-15T03:36:47.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.847+0000 7effa0cb5700 1 -- 172.21.15.110:0/1762529320 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7eff9c141700 msgr2=0x7eff9c13da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:47.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.847+0000 7effa0cb5700 1 --2- 172.21.15.110:0/1762529320 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7eff9c141700 0x7eff9c13da40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7eff94009e60 tx=0x7eff94009780 comp rx=0 tx=0).stop 2024-09-15T03:36:47.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.847+0000 7effa0cb5700 1 -- 172.21.15.110:0/1762529320 shutdown_connections 2024-09-15T03:36:47.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.847+0000 7effa0cb5700 1 --2- 172.21.15.110:0/1762529320 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7eff9c141700 0x7eff9c13da40 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:47.855 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.847+0000 7effa0cb5700 1 --2- 172.21.15.110:0/1762529320 >> v2:172.21.15.110:3300/0 conn(0x7eff9c0af4a0 0x7eff9c1411c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:47.855 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.847+0000 7effa0cb5700 1 -- 172.21.15.110:0/1762529320 >> 172.21.15.110:0/1762529320 conn(0x7eff9c067a10 msgr2=0x7eff9c067e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:47.855 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.847+0000 7effa0cb5700 1 -- 172.21.15.110:0/1762529320 shutdown_connections 2024-09-15T03:36:47.855 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.847+0000 7effa0cb5700 1 -- 172.21.15.110:0/1762529320 wait complete. 2024-09-15T03:36:47.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.851+0000 7effa0cb5700 1 Processor -- start 2024-09-15T03:36:47.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.851+0000 7effa0cb5700 1 -- start start 2024-09-15T03:36:47.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.851+0000 7effa0cb5700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7eff9c0af0d0 0x7eff9c11aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:47.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.851+0000 7effa0cb5700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7eff9c0af4a0 0x7eff9c11b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:47.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.851+0000 7effa0cb5700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff9c141700 0x7eff9c13a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:47.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.851+0000 7effa0cb5700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff9c11b690 con 0x7eff9c141700 2024-09-15T03:36:47.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.851+0000 7effa0cb5700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff9c11c820 con 0x7eff9c0af4a0 2024-09-15T03:36:47.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.851+0000 7effa0cb5700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff9c06f620 con 0x7eff9c0af0d0 2024-09-15T03:36:47.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.851+0000 7eff9b7fe700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7eff9c0af0d0 0x7eff9c11aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:47.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.851+0000 7eff9bfff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff9c141700 0x7eff9c13a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:47.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.851+0000 7eff9b7fe700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7eff9c0af0d0 0x7eff9c11aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3301/0 says I am v2:172.21.15.110:38928/0 (socket says 172.21.15.110:38928) 2024-09-15T03:36:47.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.851+0000 7eff9b7fe700 1 -- 172.21.15.110:0/3454903273 learned_addr learned my addr 172.21.15.110:0/3454903273 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:36:47.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.851+0000 7eff9b7fe700 1 -- 172.21.15.110:0/3454903273 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7eff9c0af4a0 msgr2=0x7eff9c11b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:47.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.851+0000 7eff9b7fe700 1 --2- 172.21.15.110:0/3454903273 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7eff9c0af4a0 0x7eff9c11b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:47.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.851+0000 7eff9b7fe700 1 -- 172.21.15.110:0/3454903273 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff9c141700 msgr2=0x7eff9c13a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:47.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.851+0000 7eff9b7fe700 1 --2- 172.21.15.110:0/3454903273 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff9c141700 0x7eff9c13a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:47.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.851+0000 7eff9b7fe700 1 -- 172.21.15.110:0/3454903273 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff94009a20 con 0x7eff9c0af0d0 2024-09-15T03:36:47.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.851+0000 7eff9b7fe700 1 --2- 172.21.15.110:0/3454903273 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7eff9c0af0d0 0x7eff9c11aac0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7eff8c00c5c0 tx=0x7eff8c007f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:47.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.851+0000 7eff837fe700 1 -- 172.21.15.110:0/3454903273 <== mon.2 v2:172.21.15.110:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8c005b30 con 0x7eff9c0af0d0 2024-09-15T03:36:47.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.851+0000 7eff837fe700 1 -- 172.21.15.110:0/3454903273 <== mon.2 v2:172.21.15.110:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff8c005cb0 con 0x7eff9c0af0d0 2024-09-15T03:36:47.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.851+0000 7effa0cb5700 1 -- 172.21.15.110:0/3454903273 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff9c11a5b0 con 0x7eff9c0af0d0 2024-09-15T03:36:47.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.851+0000 7eff837fe700 1 -- 172.21.15.110:0/3454903273 <== mon.2 v2:172.21.15.110:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8c007320 con 0x7eff9c0af0d0 2024-09-15T03:36:47.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.851+0000 7effa0cb5700 1 -- 172.21.15.110:0/3454903273 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff9c13a890 con 0x7eff9c0af0d0 2024-09-15T03:36:47.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.855+0000 7effa0cb5700 1 -- 172.21.15.110:0/3454903273 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff9c066dc0 con 0x7eff9c0af0d0 2024-09-15T03:36:47.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.855+0000 7eff837fe700 1 -- 172.21.15.110:0/3454903273 <== mon.2 v2:172.21.15.110:3301/0 4 ==== mgrmap(e 5) v1 ==== 85847+0+0 (secure 0 0 0) 0x7eff8c01b3b0 con 0x7eff9c0af0d0 2024-09-15T03:36:47.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.855+0000 7eff837fe700 1 --2- 172.21.15.110:0/3454903273 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7eff88067a80 0x7eff88069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:36:47.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.855+0000 7eff837fe700 1 -- 172.21.15.110:0/3454903273 <== mon.2 v2:172.21.15.110:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5583+0+0 (secure 0 0 0) 0x7eff8c084a40 con 0x7eff9c0af0d0 2024-09-15T03:36:47.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.855+0000 7eff9affd700 1 --2- 172.21.15.110:0/3454903273 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7eff88067a80 0x7eff88069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:36:47.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.855+0000 7eff9affd700 1 --2- 172.21.15.110:0/3454903273 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7eff88067a80 0x7eff88069f30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7eff9c11be40 tx=0x7eff900091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:36:47.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:47.867+0000 7eff837fe700 1 -- 172.21.15.110:0/3454903273 <== mon.2 v2:172.21.15.110:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7eff8c086e60 con 0x7eff9c0af0d0 2024-09-15T03:36:48.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:48.087+0000 7effa0cb5700 1 -- 172.21.15.110:0/3454903273 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.libcephsqlite", "caps": ["mon", "profile simple-rados-client-with-blocklist", "osd", "allow rwx pool=cephsqlite"]} v 0) v1 -- 0x7eff9c0648f0 con 0x7eff9c0af0d0 2024-09-15T03:36:48.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:48.095+0000 7eff837fe700 1 -- 172.21.15.110:0/3454903273 <== mon.2 v2:172.21.15.110:3301/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.libcephsqlite", "caps": ["mon", "profile simple-rados-client-with-blocklist", "osd", "allow rwx pool=cephsqlite"]}]=0 v4) v1 ==== 199+0+71 (secure 0 0 0) 0x7eff8c017070 con 0x7eff9c0af0d0 2024-09-15T03:36:48.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:48.099+0000 7effa0cb5700 1 -- 172.21.15.110:0/3454903273 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7eff88067a80 msgr2=0x7eff88069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:48.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:48.099+0000 7effa0cb5700 1 --2- 172.21.15.110:0/3454903273 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7eff88067a80 0x7eff88069f30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7eff9c11be40 tx=0x7eff900091a0 comp rx=0 tx=0).stop 2024-09-15T03:36:48.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:48.099+0000 7effa0cb5700 1 -- 172.21.15.110:0/3454903273 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7eff9c0af0d0 msgr2=0x7eff9c11aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:36:48.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:48.099+0000 7effa0cb5700 1 --2- 172.21.15.110:0/3454903273 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7eff9c0af0d0 0x7eff9c11aac0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7eff8c00c5c0 tx=0x7eff8c007f90 comp rx=0 tx=0).stop 2024-09-15T03:36:48.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:48.099+0000 7effa0cb5700 1 -- 172.21.15.110:0/3454903273 shutdown_connections 2024-09-15T03:36:48.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:48.099+0000 7effa0cb5700 1 --2- 172.21.15.110:0/3454903273 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7eff88067a80 0x7eff88069f30 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:48.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:48.099+0000 7effa0cb5700 1 --2- 172.21.15.110:0/3454903273 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff9c141700 0x7eff9c13a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:48.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:48.099+0000 7effa0cb5700 1 --2- 172.21.15.110:0/3454903273 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7eff9c0af4a0 0x7eff9c11b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:48.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:48.099+0000 7effa0cb5700 1 --2- 172.21.15.110:0/3454903273 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7eff9c0af0d0 0x7eff9c11aac0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:36:48.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:48.099+0000 7effa0cb5700 1 -- 172.21.15.110:0/3454903273 >> 172.21.15.110:0/3454903273 conn(0x7eff9c067a10 msgr2=0x7eff9c0afc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:36:48.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:48.099+0000 7effa0cb5700 1 -- 172.21.15.110:0/3454903273 shutdown_connections 2024-09-15T03:36:48.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:36:48.099+0000 7effa0cb5700 1 -- 172.21.15.110:0/3454903273 wait complete. 2024-09-15T03:36:48.117 INFO:teuthology.run_tasks:Running task exec... 2024-09-15T03:36:48.126 INFO:teuthology.task.exec:Executing custom commands... 2024-09-15T03:36:48.126 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi110.front.sepia.ceph.com 2024-09-15T03:36:48.126 DEBUG:teuthology.orchestra.run.smithi110:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph_test_libcephsqlite --id libcephsqlite --no-log-to-stderr' 2024-09-15T03:36:48.164 INFO:teuthology.orchestra.run.smithi110.stdout:[==========] Running 20 tests from 1 test suite. 2024-09-15T03:36:48.164 INFO:teuthology.orchestra.run.smithi110.stdout:[----------] Global test environment set-up. 2024-09-15T03:36:48.164 INFO:teuthology.orchestra.run.smithi110.stdout:[----------] 20 tests from CephSQLiteTest 2024-09-15T03:36:53.174 INFO:teuthology.orchestra.run.smithi110.stdout:[ RUN ] CephSQLiteTest.Create 2024-09-15T03:36:53.196 INFO:teuthology.orchestra.run.smithi110.stdout:using database: file:cephsqlite:/fed4b1a3-8572-468c-9671-91e903ab139c.db?vfs=ceph 2024-09-15T03:36:53.196 INFO:teuthology.orchestra.run.smithi110.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-09-15T03:36:53.206 INFO:teuthology.orchestra.run.smithi110.stdout:CREATE TABLE foo (a INT); 2024-09-15T03:36:53.239 INFO:teuthology.orchestra.run.smithi110.stdout:[ OK ] CephSQLiteTest.Create (66 ms) 2024-09-15T03:36:53.240 INFO:teuthology.orchestra.run.smithi110.stdout:[ RUN ] CephSQLiteTest.InsertBulk4096 2024-09-15T03:36:53.262 INFO:teuthology.orchestra.run.smithi110.stdout:using database: file:cephsqlite:/0a627d6a-d085-487d-8743-618e0f4c239e.db?vfs=ceph 2024-09-15T03:36:53.262 INFO:teuthology.orchestra.run.smithi110.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-09-15T03:36:53.478 INFO:teuthology.orchestra.run.smithi110.stdout:PRAGMA page_size = 4096;CREATE TABLE foo (a INT);WITH RECURSIVE c(x) AS ( VALUES(1) UNION ALL SELECT x+1 FROM c )INSERT INTO foo (a) SELECT RANDOM() FROM c LIMIT 1000000;PRAGMA page_size; 2024-09-15T03:36:55.475 INFO:teuthology.orchestra.run.smithi110.stdout:[ OK ] CephSQLiteTest.InsertBulk4096 (2236 ms) 2024-09-15T03:36:55.475 INFO:teuthology.orchestra.run.smithi110.stdout:[ RUN ] CephSQLiteTest.InsertBulk 2024-09-15T03:36:55.490 INFO:teuthology.orchestra.run.smithi110.stdout:using database: file:cephsqlite:/15e74950-6e70-4567-bff3-a2024c588311.db?vfs=ceph 2024-09-15T03:36:55.490 INFO:teuthology.orchestra.run.smithi110.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-09-15T03:36:55.500 INFO:teuthology.orchestra.run.smithi110.stdout:CREATE TABLE foo (a INT);WITH RECURSIVE c(x) AS ( VALUES(1) UNION ALL SELECT x+1 FROM c )INSERT INTO foo (a) SELECT RANDOM() FROM c LIMIT 1000000; 2024-09-15T03:36:56.041 INFO:teuthology.orchestra.run.smithi110.stdout:[ OK ] CephSQLiteTest.InsertBulk (565 ms) 2024-09-15T03:36:56.041 INFO:teuthology.orchestra.run.smithi110.stdout:[ RUN ] CephSQLiteTest.UpdateBulk 2024-09-15T03:36:56.056 INFO:teuthology.orchestra.run.smithi110.stdout:using database: file:cephsqlite:/8332059d-b895-44c5-a57a-8cb73acc258a.db?vfs=ceph 2024-09-15T03:36:56.056 INFO:teuthology.orchestra.run.smithi110.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-09-15T03:36:56.063 INFO:teuthology.orchestra.run.smithi110.stdout:CREATE TABLE foo (a INT);WITH RECURSIVE c(x) AS ( VALUES(1) UNION ALL SELECT x+1 FROM c )INSERT INTO foo (a) SELECT x FROM c LIMIT 1000000;SELECT SUM(a) FROM foo;UPDATE foo SET a = a+a;SELECT SUM(a) FROM foo; 2024-09-15T03:36:57.012 INFO:teuthology.orchestra.run.smithi110.stdout:[ OK ] CephSQLiteTest.UpdateBulk (972 ms) 2024-09-15T03:36:57.012 INFO:teuthology.orchestra.run.smithi110.stdout:[ RUN ] CephSQLiteTest.InsertRate 2024-09-15T03:36:57.030 INFO:teuthology.orchestra.run.smithi110.stdout:using database: file:cephsqlite:/0414edde-6f68-4253-aa8a-d5ff59b0af33.db?vfs=ceph 2024-09-15T03:36:57.030 INFO:teuthology.orchestra.run.smithi110.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-09-15T03:36:57.037 INFO:teuthology.orchestra.run.smithi110.stdout:CREATE TABLE foo (a INT);INSERT INTO foo (a) VALUES (RANDOM()); 2024-09-15T03:36:59.882 INFO:teuthology.orchestra.run.smithi110.stdout:transactions per second: 35.5627 2024-09-15T03:36:59.883 INFO:teuthology.orchestra.run.smithi110.stdout:[ OK ] CephSQLiteTest.InsertRate (2870 ms) 2024-09-15T03:36:59.883 INFO:teuthology.orchestra.run.smithi110.stdout:[ RUN ] CephSQLiteTest.DatabaseShrink 2024-09-15T03:36:59.902 INFO:teuthology.orchestra.run.smithi110.stdout:using database: file:cephsqlite:/d4ba3007-f0f1-40ed-a5d2-1afd7e411884.db?vfs=ceph 2024-09-15T03:36:59.902 INFO:teuthology.orchestra.run.smithi110.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-09-15T03:36:59.911 INFO:teuthology.orchestra.run.smithi110.stdout:CREATE TABLE foo (a INT);WITH RECURSIVE c(x) AS ( VALUES(1) UNION ALL SELECT x+1 FROM c )INSERT INTO foo (a) SELECT x FROM c LIMIT 1000000;DELETE FROM foo WHERE RANDOM()%4 < 3;VACUUM; 2024-09-15T03:37:01.430 INFO:teuthology.orchestra.run.smithi110.stdout:[ OK ] CephSQLiteTest.DatabaseShrink (1546 ms) 2024-09-15T03:37:01.430 INFO:teuthology.orchestra.run.smithi110.stdout:[ RUN ] CephSQLiteTest.InsertExclusiveRate 2024-09-15T03:37:01.448 INFO:teuthology.orchestra.run.smithi110.stdout:using database: file:cephsqlite:/dcd978b2-802d-49b6-83df-6ed9dc6b272d.db?vfs=ceph 2024-09-15T03:37:01.448 INFO:teuthology.orchestra.run.smithi110.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-09-15T03:37:01.457 INFO:teuthology.orchestra.run.smithi110.stdout:PRAGMA locking_mode=EXCLUSIVE;CREATE TABLE foo (a INT);INSERT INTO foo (a) VALUES (RANDOM()); 2024-09-15T03:37:02.787 INFO:teuthology.orchestra.run.smithi110.stdout:transactions per second: 76.6889 2024-09-15T03:37:02.791 INFO:teuthology.orchestra.run.smithi110.stdout:[ OK ] CephSQLiteTest.InsertExclusiveRate (1362 ms) 2024-09-15T03:37:02.791 INFO:teuthology.orchestra.run.smithi110.stdout:[ RUN ] CephSQLiteTest.InsertExclusiveWALRate 2024-09-15T03:37:02.810 INFO:teuthology.orchestra.run.smithi110.stdout:using database: file:cephsqlite:/ff1e30ff-9b88-42f2-9102-39df792c6c34.db?vfs=ceph 2024-09-15T03:37:02.810 INFO:teuthology.orchestra.run.smithi110.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-09-15T03:37:02.817 INFO:teuthology.orchestra.run.smithi110.stdout:PRAGMA locking_mode=EXCLUSIVE;PRAGMA journal_mode=WAL;CREATE TABLE foo (a INT);INSERT INTO foo (a) VALUES (RANDOM()); 2024-09-15T03:37:03.343 INFO:teuthology.orchestra.run.smithi110.stdout:transactions per second: 208.338 2024-09-15T03:37:03.375 INFO:teuthology.orchestra.run.smithi110.stdout:[ OK ] CephSQLiteTest.InsertExclusiveWALRate (583 ms) 2024-09-15T03:37:03.375 INFO:teuthology.orchestra.run.smithi110.stdout:[ RUN ] CephSQLiteTest.WALTransactionSync 2024-09-15T03:37:03.389 INFO:teuthology.orchestra.run.smithi110.stdout:using database: file:cephsqlite:/9a7144ff-c2b4-4aed-a383-56323ab13b43.db?vfs=ceph 2024-09-15T03:37:03.389 INFO:teuthology.orchestra.run.smithi110.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-09-15T03:37:03.396 INFO:teuthology.orchestra.run.smithi110.stdout:PRAGMA locking_mode=EXCLUSIVE;PRAGMA journal_mode=WAL;CREATE TABLE foo (a INT);INSERT INTO perf (v) VALUES (ceph_perf());BEGIN TRANSACTION;INSERT INTO foo (a) VALUES (RANDOM());END TRANSACTION;INSERT INTO perf (v) VALUES (ceph_perf());SELECT a.atom-b.atom FROM p AS a, p AS b WHERE a.i = ? AND b.i = ? AND a.fullkey = '$.libcephsqlite_vfs.opf_sync.avgcount' AND b.fullkey = '$.libcephsqlite_vfs.opf_sync.avgcount'; 2024-09-15T03:37:03.488 INFO:teuthology.orchestra.run.smithi110.stdout:[ OK ] CephSQLiteTest.WALTransactionSync (114 ms) 2024-09-15T03:37:03.489 INFO:teuthology.orchestra.run.smithi110.stdout:[ RUN ] CephSQLiteTest.PersistTransactionSync 2024-09-15T03:37:03.501 INFO:teuthology.orchestra.run.smithi110.stdout:using database: file:cephsqlite:/e6d457f9-a29d-4ca3-a0c3-4c23a461f36f.db?vfs=ceph 2024-09-15T03:37:03.501 INFO:teuthology.orchestra.run.smithi110.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-09-15T03:37:03.507 INFO:teuthology.orchestra.run.smithi110.stdout:BEGIN TRANSACTION;CREATE TABLE foo (a INT);INSERT INTO foo (a) VALUES (RANDOM());END TRANSACTION;INSERT INTO perf (v) VALUES (ceph_perf());SELECT a.atom-b.atom FROM p AS a, p AS b WHERE a.i = ? AND b.i = ? AND a.fullkey = '$.libcephsqlite_vfs.opf_sync.avgcount' AND b.fullkey = '$.libcephsqlite_vfs.opf_sync.avgcount'; 2024-09-15T03:37:03.538 INFO:teuthology.orchestra.run.smithi110.stdout:[ OK ] CephSQLiteTest.PersistTransactionSync (50 ms) 2024-09-15T03:37:03.538 INFO:teuthology.orchestra.run.smithi110.stdout:[ RUN ] CephSQLiteTest.InsertExclusiveLock 2024-09-15T03:37:03.554 INFO:teuthology.orchestra.run.smithi110.stdout:using database: file:cephsqlite:/10b79af9-3949-4d24-b9dd-63c1c03a74ac.db?vfs=ceph 2024-09-15T03:37:03.554 INFO:teuthology.orchestra.run.smithi110.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-09-15T03:37:03.562 INFO:teuthology.orchestra.run.smithi110.stdout:PRAGMA locking_mode=EXCLUSIVE;CREATE TABLE foo (a INT);INSERT INTO foo (a) VALUES (RANDOM());INSERT INTO perf (v) VALUES (ceph_perf());SELECT a.atom, b.atom, a.atom-b.atom FROM p AS a, p AS b WHERE a.i = ? AND b.i = ? AND a.fullkey = '$.libcephsqlite_vfs.opf_lock.avgcount' AND b.fullkey = '$.libcephsqlite_vfs.opf_lock.avgcount';SELECT a.atom, b.atom, a.atom-b.atom FROM p AS a, p AS b WHERE a.i = ? AND b.i = ? AND a.fullkey = '$.libcephsqlite_striper.lock' AND b.fullkey = '$.libcephsqlite_striper.lock'; 2024-09-15T03:37:03.602 INFO:teuthology.orchestra.run.smithi110.stdout:[ OK ] CephSQLiteTest.InsertExclusiveLock (63 ms) 2024-09-15T03:37:03.602 INFO:teuthology.orchestra.run.smithi110.stdout:[ RUN ] CephSQLiteTest.TransactionSizeUpdate 2024-09-15T03:37:03.619 INFO:teuthology.orchestra.run.smithi110.stdout:using database: file:cephsqlite:/3e4989a9-0ab1-4cfd-903e-eed4db72119e.db?vfs=ceph 2024-09-15T03:37:03.619 INFO:teuthology.orchestra.run.smithi110.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-09-15T03:37:03.630 INFO:teuthology.orchestra.run.smithi110.stdout:BEGIN TRANSACTION;CREATE TABLE foo (a INT);INSERT INTO foo (a) VALUES (RANDOM());END TRANSACTION;INSERT INTO perf (v) VALUES (ceph_perf());SELECT a.atom, b.atom, a.atom-b.atom FROM p AS a, p AS b WHERE a.i = ? AND b.i = ? AND a.fullkey = '$.libcephsqlite_striper.update_size' AND b.fullkey = '$.libcephsqlite_striper.update_size'; 2024-09-15T03:37:03.674 INFO:teuthology.orchestra.run.smithi110.stdout:[ OK ] CephSQLiteTest.TransactionSizeUpdate (73 ms) 2024-09-15T03:37:03.675 INFO:teuthology.orchestra.run.smithi110.stdout:[ RUN ] CephSQLiteTest.AllocatedGrowth 2024-09-15T03:37:03.694 INFO:teuthology.orchestra.run.smithi110.stdout:using database: file:cephsqlite:/57479495-368f-432e-8e80-88d6da31e9bc.db?vfs=ceph 2024-09-15T03:37:03.694 INFO:teuthology.orchestra.run.smithi110.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-09-15T03:37:03.703 INFO:teuthology.orchestra.run.smithi110.stdout:CREATE TABLE foo (a BLOB);WITH RECURSIVE c(x) AS ( VALUES(1) UNION ALL SELECT x+1 FROM c )INSERT INTO foo (a) SELECT RANDOMBLOB(1<<20) FROM c LIMIT 1024;INSERT INTO perf (v) VALUES (ceph_perf());SELECT a.atom, b.atom, a.atom-b.atom FROM p AS a, p AS b WHERE a.i = ? AND b.i = ? AND a.fullkey = '$.libcephsqlite_striper.update_allocated' AND b.fullkey = '$.libcephsqlite_striper.update_allocated'; 2024-09-15T03:37:10.390 INFO:teuthology.orchestra.run.smithi110.stdout:[ OK ] CephSQLiteTest.AllocatedGrowth (6715 ms) 2024-09-15T03:37:10.390 INFO:teuthology.orchestra.run.smithi110.stdout:[ RUN ] CephSQLiteTest.DeleteBulk 2024-09-15T03:37:10.404 INFO:teuthology.orchestra.run.smithi110.stdout:using database: file:cephsqlite:/a417a44c-8228-4d65-80eb-25213a2e6e9c.db?vfs=ceph 2024-09-15T03:37:10.405 INFO:teuthology.orchestra.run.smithi110.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-09-15T03:37:10.413 INFO:teuthology.orchestra.run.smithi110.stdout:CREATE TABLE foo (a INT);WITH RECURSIVE c(x) AS ( VALUES(1) UNION ALL SELECT x+1 FROM c )INSERT INTO foo (a) SELECT x FROM c LIMIT 1000000;DELETE FROM foo WHERE RANDOM()%2 == 0; 2024-09-15T03:37:11.275 INFO:teuthology.orchestra.run.smithi110.stdout:[ OK ] CephSQLiteTest.DeleteBulk (885 ms) 2024-09-15T03:37:11.275 INFO:teuthology.orchestra.run.smithi110.stdout:[ RUN ] CephSQLiteTest.DropMassive 2024-09-15T03:37:11.293 INFO:teuthology.orchestra.run.smithi110.stdout:using database: file:cephsqlite:/18a73417-1aff-4ceb-867f-72eb6fc83ff4.db?vfs=ceph 2024-09-15T03:37:11.294 INFO:teuthology.orchestra.run.smithi110.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-09-15T03:37:11.301 INFO:teuthology.orchestra.run.smithi110.stdout:CREATE TABLE foo (a BLOB);WITH RECURSIVE c(x) AS ( VALUES(1) UNION ALL SELECT x+1 FROM c )INSERT INTO foo (a) SELECT RANDOMBLOB(1<<20) FROM c LIMIT 1024;DROP TABLE foo;VACUUM;INSERT INTO perf (v) VALUES (ceph_perf());SELECT a.atom, b.atom FROM p AS a, p AS b WHERE a.i = ? AND b.i = ? AND a.fullkey = '$.libcephsqlite_striper.shrink' AND b.fullkey = '$.libcephsqlite_striper.shrink';SELECT a.atom-b.atom FROM p AS a, p AS b WHERE a.i = ? AND b.i = ? AND a.fullkey = '$.libcephsqlite_striper.shrink_bytes' AND b.fullkey = '$.libcephsqlite_striper.shrink_bytes'; 2024-09-15T03:37:43.976 INFO:teuthology.orchestra.run.smithi110.stdout:[ OK ] CephSQLiteTest.DropMassive (32702 ms) 2024-09-15T03:37:43.977 INFO:teuthology.orchestra.run.smithi110.stdout:[ RUN ] CephSQLiteTest.InsertMassiveVerify 2024-09-15T03:37:43.992 INFO:teuthology.orchestra.run.smithi110.stdout:using database: file:cephsqlite:/52968886-1167-415d-80be-6e48c3d4132a.db?vfs=ceph 2024-09-15T03:37:43.992 INFO:teuthology.orchestra.run.smithi110.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-09-15T03:37:44.003 INFO:teuthology.orchestra.run.smithi110.stdout:CREATE TABLE foo (a BLOB);CREATE TEMPORARY TABLE bar (a BLOB);WITH RECURSIVE c(x) AS ( VALUES(1) UNION ALL SELECT x+1 FROM c )INSERT INTO bar (a) SELECT RANDOMBLOB(1<<20) FROM c LIMIT 1024;SELECT a FROM bar;INSERT INTO foo (a) SELECT a FROM bar;SELECT a FROM foo; 2024-09-15T03:37:58.157 INFO:teuthology.orchestra.run.smithi110.stdout:[ OK ] CephSQLiteTest.InsertMassiveVerify (14180 ms) 2024-09-15T03:37:58.157 INFO:teuthology.orchestra.run.smithi110.stdout:[ RUN ] CephSQLiteTest.PerfValid 2024-09-15T03:37:58.173 INFO:teuthology.orchestra.run.smithi110.stdout:using database: file:cephsqlite:/21629216-0a19-4356-b3fa-d0ceb558106b.db?vfs=ceph 2024-09-15T03:37:58.173 INFO:teuthology.orchestra.run.smithi110.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-09-15T03:37:58.183 INFO:teuthology.orchestra.run.smithi110.stdout:SELECT json_valid(ceph_perf()); 2024-09-15T03:37:58.184 INFO:teuthology.orchestra.run.smithi110.stdout:[ OK ] CephSQLiteTest.PerfValid (28 ms) 2024-09-15T03:37:58.184 INFO:teuthology.orchestra.run.smithi110.stdout:[ RUN ] CephSQLiteTest.StatusValid 2024-09-15T03:37:58.199 INFO:teuthology.orchestra.run.smithi110.stdout:using database: file:cephsqlite:/3297615e-bca4-4f1f-b391-942a7d75adf9.db?vfs=ceph 2024-09-15T03:37:58.199 INFO:teuthology.orchestra.run.smithi110.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-09-15T03:37:58.211 INFO:teuthology.orchestra.run.smithi110.stdout:SELECT json_valid(ceph_status()); 2024-09-15T03:37:58.212 INFO:teuthology.orchestra.run.smithi110.stdout:[ OK ] CephSQLiteTest.StatusValid (29 ms) 2024-09-15T03:37:58.212 INFO:teuthology.orchestra.run.smithi110.stdout:[ RUN ] CephSQLiteTest.CurrentTime 2024-09-15T03:37:58.232 INFO:teuthology.orchestra.run.smithi110.stdout:using database: file:cephsqlite:/7075c230-68d1-4f5f-ada9-1c743ba31eb4.db?vfs=ceph 2024-09-15T03:37:58.232 INFO:teuthology.orchestra.run.smithi110.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-09-15T03:37:58.244 INFO:teuthology.orchestra.run.smithi110.stdout:SELECT strftime('%s', 'now'); 2024-09-15T03:37:58.245 INFO:teuthology.orchestra.run.smithi110.stdout:[ OK ] CephSQLiteTest.CurrentTime (32 ms) 2024-09-15T03:37:58.245 INFO:teuthology.orchestra.run.smithi110.stdout:[ RUN ] CephSQLiteTest.StatusFields 2024-09-15T03:37:58.265 INFO:teuthology.orchestra.run.smithi110.stdout:using database: file:cephsqlite:/cad46367-38c8-45b3-9d7d-1996e8bb6544.db?vfs=ceph 2024-09-15T03:37:58.266 INFO:teuthology.orchestra.run.smithi110.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-09-15T03:37:58.278 INFO:teuthology.orchestra.run.smithi110.stdout:SELECT json_extract(ceph_status(), '$.addr');SELECT json_extract(ceph_status(), '$.id'); 2024-09-15T03:37:58.278 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/1603257054 2024-09-15T03:37:58.279 INFO:teuthology.orchestra.run.smithi110.stdout:4376 2024-09-15T03:37:58.280 INFO:teuthology.orchestra.run.smithi110.stdout:[ OK ] CephSQLiteTest.StatusFields (35 ms) 2024-09-15T03:37:58.280 INFO:teuthology.orchestra.run.smithi110.stdout:[----------] 20 tests from CephSQLiteTest (65107 ms total) 2024-09-15T03:37:58.280 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-15T03:37:58.280 INFO:teuthology.orchestra.run.smithi110.stdout:[----------] Global test environment tear-down 2024-09-15T03:37:58.280 INFO:teuthology.orchestra.run.smithi110.stdout:[==========] 20 tests from 1 test suite ran. (70116 ms total) 2024-09-15T03:37:58.280 INFO:teuthology.orchestra.run.smithi110.stdout:[ PASSED ] 20 tests. 2024-09-15T03:37:58.322 INFO:teuthology.run_tasks:Running task workunit... 2024-09-15T03:37:58.343 INFO:tasks.workunit:Pulling workunits from ref d75f23820806da622f561ee5ee7551015285993f 2024-09-15T03:37:58.344 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-15T03:37:58.344 DEBUG:teuthology.orchestra.run.smithi110:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-15T03:37:58.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T03:37:58.351 INFO:teuthology.orchestra.run.smithi110.stderr:stat: cannot stat '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-15T03:37:58.352 DEBUG:teuthology.orchestra.run.smithi110:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-09-15T03:37:58.376 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-09-15T03:37:58.376 DEBUG:teuthology.orchestra.run.smithi110:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-09-15T03:37:58.423 INFO:tasks.workunit:timeout=3h 2024-09-15T03:37:58.423 INFO:tasks.workunit:cleanup=True 2024-09-15T03:37:58.424 DEBUG:teuthology.orchestra.run.smithi110:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout d75f23820806da622f561ee5ee7551015285993f 2024-09-15T03:37:58.432 INFO:tasks.workunit.client.0.smithi110.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-15T03:38:49.551 INFO:tasks.workunit.client.0.smithi110.stderr:Note: switching to 'd75f23820806da622f561ee5ee7551015285993f'. 2024-09-15T03:38:49.552 INFO:tasks.workunit.client.0.smithi110.stderr: 2024-09-15T03:38:49.552 INFO:tasks.workunit.client.0.smithi110.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-15T03:38:49.552 INFO:tasks.workunit.client.0.smithi110.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-15T03:38:49.552 INFO:tasks.workunit.client.0.smithi110.stderr:state without impacting any branches by switching back to a branch. 2024-09-15T03:38:49.552 INFO:tasks.workunit.client.0.smithi110.stderr: 2024-09-15T03:38:49.552 INFO:tasks.workunit.client.0.smithi110.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-15T03:38:49.552 INFO:tasks.workunit.client.0.smithi110.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-15T03:38:49.553 INFO:tasks.workunit.client.0.smithi110.stderr: 2024-09-15T03:38:49.553 INFO:tasks.workunit.client.0.smithi110.stderr: git switch -c 2024-09-15T03:38:49.553 INFO:tasks.workunit.client.0.smithi110.stderr: 2024-09-15T03:38:49.553 INFO:tasks.workunit.client.0.smithi110.stderr:Or undo this operation with: 2024-09-15T03:38:49.553 INFO:tasks.workunit.client.0.smithi110.stderr: 2024-09-15T03:38:49.553 INFO:tasks.workunit.client.0.smithi110.stderr: git switch - 2024-09-15T03:38:49.553 INFO:tasks.workunit.client.0.smithi110.stderr: 2024-09-15T03:38:49.553 INFO:tasks.workunit.client.0.smithi110.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-15T03:38:49.553 INFO:tasks.workunit.client.0.smithi110.stderr: 2024-09-15T03:38:49.553 INFO:tasks.workunit.client.0.smithi110.stderr:HEAD is now at d75f2382080 Merge branch 'wip-67762-quincy' of https://github.com/nbalacha/ceph into wip-bharath9-testing-2024-09-13-1444-quincy 2024-09-15T03:38:49.557 DEBUG:teuthology.orchestra.run.smithi110:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-15T03:38:49.585 INFO:tasks.workunit.client.0.smithi110.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-15T03:38:49.588 INFO:tasks.workunit.client.0.smithi110.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-15T03:38:49.588 INFO:tasks.workunit.client.0.smithi110.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-15T03:38:50.467 INFO:tasks.workunit.client.0.smithi110.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-15T03:38:50.575 INFO:tasks.workunit.client.0.smithi110.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-15T03:38:50.634 INFO:tasks.workunit.client.0.smithi110.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-15T03:38:50.636 INFO:tasks.workunit.client.0.smithi110.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-15T03:38:50.637 INFO:tasks.workunit.client.0.smithi110.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-15T03:38:50.698 INFO:tasks.workunit.client.0.smithi110.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-15T03:38:50.703 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-15T03:38:50.703 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-15T03:38:50.709 INFO:tasks.workunit:Running workunits matching rados/test_libcephsqlite.sh on client.0... 2024-09-15T03:38:50.710 INFO:tasks.workunit:Running workunit rados/test_libcephsqlite.sh... 2024-09-15T03:38:50.711 DEBUG:teuthology.orchestra.run.smithi110:workunit test rados/test_libcephsqlite.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d75f23820806da622f561ee5ee7551015285993f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 CEPH_ARGS='--id libcephsqlite --no-log-to-stderr' adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_libcephsqlite.sh cephsqlite 2024-09-15T03:38:50.723 INFO:tasks.workunit.client.0.smithi110.stderr:+ pool=cephsqlite 2024-09-15T03:38:50.724 INFO:tasks.workunit.client.0.smithi110.stderr:++ basename /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_libcephsqlite.sh 2024-09-15T03:38:50.727 INFO:tasks.workunit.client.0.smithi110.stderr:+ ns=test_libcephsqlite.sh 2024-09-15T03:38:50.727 INFO:tasks.workunit.client.0.smithi110.stderr:+ striper rm baz.db 2024-09-15T03:38:50.727 INFO:tasks.workunit.client.0.smithi110.stderr:+ rados --pool=cephsqlite --namespace=test_libcephsqlite.sh --striper rm baz.db 2024-09-15T03:38:50.786 INFO:tasks.workunit.client.0.smithi110.stderr:+ sqlite 2024-09-15T03:38:50.786 INFO:tasks.workunit.client.0.smithi110.stderr:+ background= 2024-09-15T03:38:50.786 INFO:tasks.workunit.client.0.smithi110.stderr:+ '[' '' = b ']' 2024-09-15T03:38:50.787 INFO:tasks.workunit.client.0.smithi110.stderr:++ cat 2024-09-15T03:38:50.788 INFO:tasks.workunit.client.0.smithi110.stderr:+ a='create table if not exists foo (a INT); 2024-09-15T03:38:50.788 INFO:tasks.workunit.client.0.smithi110.stderr:insert into foo (a) values (RANDOM()); 2024-09-15T03:38:50.788 INFO:tasks.workunit.client.0.smithi110.stderr:drop table foo;' 2024-09-15T03:38:50.788 INFO:tasks.workunit.client.0.smithi110.stderr:+ printf %s 'create table if not exists foo (a INT); 2024-09-15T03:38:50.788 INFO:tasks.workunit.client.0.smithi110.stderr:insert into foo (a) values (RANDOM()); 2024-09-15T03:38:50.788 INFO:tasks.workunit.client.0.smithi110.stderr:drop table foo;' 2024-09-15T03:38:50.788 INFO:tasks.workunit.client.0.smithi110.stderr:create table if not exists foo (a INT); 2024-09-15T03:38:50.789 INFO:tasks.workunit.client.0.smithi110.stderr:insert into foo (a) values (RANDOM()); 2024-09-15T03:38:50.789 INFO:tasks.workunit.client.0.smithi110.stderr:drop table foo;+ '[' '' '!=' b ']' 2024-09-15T03:38:50.789 INFO:tasks.workunit.client.0.smithi110.stderr:+ wait 2024-09-15T03:38:50.789 INFO:tasks.workunit.client.0.smithi110.stderr:+ sqlite3 -cmd '.output /dev/null' -cmd '.load libcephsqlite.so' -cmd 'pragma journal_mode = PERSIST' -cmd '.open file:///cephsqlite:test_libcephsqlite.sh/baz.db?vfs=ceph' -cmd '.output stdout' 2024-09-15T03:38:51.087 INFO:tasks.workunit.client.0.smithi110.stderr: 2024-09-15T03:38:51.087 INFO:tasks.workunit.client.0.smithi110.stderr:real 0m0.301s 2024-09-15T03:38:51.087 INFO:tasks.workunit.client.0.smithi110.stderr:user 0m0.066s 2024-09-15T03:38:51.087 INFO:tasks.workunit.client.0.smithi110.stderr:sys 0m0.039s 2024-09-15T03:38:51.087 INFO:tasks.workunit.client.0.smithi110.stderr:+ striper stat baz.db 2024-09-15T03:38:51.087 INFO:tasks.workunit.client.0.smithi110.stderr:+ rados --pool=cephsqlite --namespace=test_libcephsqlite.sh --striper stat baz.db 2024-09-15T03:38:51.129 INFO:tasks.workunit.client.0.smithi110.stdout:cephsqlite/baz.db mtime 2024-09-15T03:38:51.000000+0000, size 8192 2024-09-15T03:38:51.132 INFO:tasks.workunit.client.0.smithi110.stderr:+ striper rm baz.db 2024-09-15T03:38:51.132 INFO:tasks.workunit.client.0.smithi110.stderr:+ rados --pool=cephsqlite --namespace=test_libcephsqlite.sh --striper rm baz.db 2024-09-15T03:38:51.190 INFO:tasks.workunit.client.0.smithi110.stderr:+ sqlite 2024-09-15T03:38:51.191 INFO:tasks.workunit.client.0.smithi110.stderr:++ repeat 10 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:51.191 INFO:tasks.workunit.client.0.smithi110.stderr:++ n=10 2024-09-15T03:38:51.191 INFO:tasks.workunit.client.0.smithi110.stderr:++ shift 2024-09-15T03:38:51.192 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i = 0 )) 2024-09-15T03:38:51.192 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 10 )) 2024-09-15T03:38:51.192 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:51.192 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:51.192 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 10 )) 2024-09-15T03:38:51.192 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:51.192 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:51.192 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 10 )) 2024-09-15T03:38:51.192 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:51.193 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:51.193 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 10 )) 2024-09-15T03:38:51.193 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:51.193 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:51.193 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 10 )) 2024-09-15T03:38:51.193 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:51.193 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:51.193 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 10 )) 2024-09-15T03:38:51.193 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:51.193 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:51.193 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 10 )) 2024-09-15T03:38:51.193 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:51.193 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:51.194 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 10 )) 2024-09-15T03:38:51.194 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:51.195 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:51.195 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 10 )) 2024-09-15T03:38:51.195 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:51.195 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:51.195 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 10 )) 2024-09-15T03:38:51.195 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:51.196 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:51.196 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 10 )) 2024-09-15T03:38:51.196 INFO:tasks.workunit.client.0.smithi110.stderr:+ background= 2024-09-15T03:38:51.196 INFO:tasks.workunit.client.0.smithi110.stderr:+ '[' '' = b ']' 2024-09-15T03:38:51.196 INFO:tasks.workunit.client.0.smithi110.stderr:++ cat 2024-09-15T03:38:51.196 INFO:tasks.workunit.client.0.smithi110.stderr:+ a='CREATE TABLE IF NOT EXISTS rand(text BLOB NOT NULL); 2024-09-15T03:38:51.196 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:51.196 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:51.196 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:51.196 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:51.196 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:51.196 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:51.197 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:51.197 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:51.197 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:51.197 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:51.197 INFO:tasks.workunit.client.0.smithi110.stderr:SELECT LENGTH(text) FROM rand; 2024-09-15T03:38:51.197 INFO:tasks.workunit.client.0.smithi110.stderr:DROP TABLE rand;' 2024-09-15T03:38:51.197 INFO:tasks.workunit.client.0.smithi110.stderr:+ printf %s 'CREATE TABLE IF NOT EXISTS rand(text BLOB NOT NULL); 2024-09-15T03:38:51.197 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:51.197 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:51.197 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:51.197 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:51.197 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:51.197 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:51.198 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:51.198 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:51.198 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:51.198 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:51.198 INFO:tasks.workunit.client.0.smithi110.stderr:SELECT LENGTH(text) FROM rand; 2024-09-15T03:38:51.198 INFO:tasks.workunit.client.0.smithi110.stderr:DROP TABLE rand;' 2024-09-15T03:38:51.198 INFO:tasks.workunit.client.0.smithi110.stderr:CREATE TABLE IF NOT EXISTS rand(text BLOB NOT NULL); 2024-09-15T03:38:51.198 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:51.198 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:51.198 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:51.198 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:51.198 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:51.198 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:51.199 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:51.199 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:51.199 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:51.199 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:51.199 INFO:tasks.workunit.client.0.smithi110.stderr:SELECT LENGTH(text) FROM rand; 2024-09-15T03:38:51.199 INFO:tasks.workunit.client.0.smithi110.stderr:DROP TABLE rand;+ '[' '' '!=' b ']' 2024-09-15T03:38:51.199 INFO:tasks.workunit.client.0.smithi110.stderr:+ wait 2024-09-15T03:38:51.199 INFO:tasks.workunit.client.0.smithi110.stderr:+ sqlite3 -cmd '.output /dev/null' -cmd '.load libcephsqlite.so' -cmd 'pragma journal_mode = PERSIST' -cmd '.open file:///cephsqlite:test_libcephsqlite.sh/baz.db?vfs=ceph' -cmd '.output stdout' 2024-09-15T03:38:52.152 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.153 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.153 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.153 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.153 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.153 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.153 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.153 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.153 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.153 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.335 INFO:tasks.workunit.client.0.smithi110.stderr: 2024-09-15T03:38:52.335 INFO:tasks.workunit.client.0.smithi110.stderr:real 0m1.142s 2024-09-15T03:38:52.335 INFO:tasks.workunit.client.0.smithi110.stderr:user 0m0.158s 2024-09-15T03:38:52.335 INFO:tasks.workunit.client.0.smithi110.stderr:sys 0m0.095s 2024-09-15T03:38:52.336 INFO:tasks.workunit.client.0.smithi110.stderr:+ sqlite 2024-09-15T03:38:52.336 INFO:tasks.workunit.client.0.smithi110.stderr:++ repeat 100 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.336 INFO:tasks.workunit.client.0.smithi110.stderr:++ n=100 2024-09-15T03:38:52.336 INFO:tasks.workunit.client.0.smithi110.stderr:++ shift 2024-09-15T03:38:52.336 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i = 0 )) 2024-09-15T03:38:52.336 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.336 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.336 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.336 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.336 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.336 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.336 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.337 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.337 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.337 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.337 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.337 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.337 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.337 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.338 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.338 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.338 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.338 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.338 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.338 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.338 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.338 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.339 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.339 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.339 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.339 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.339 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.339 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.339 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.339 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.339 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.339 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.339 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.339 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.339 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.340 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.340 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.340 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.340 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.340 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.340 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.340 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.340 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.340 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.340 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.340 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.340 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.340 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.341 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.341 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.341 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.359 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.359 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.359 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.359 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.359 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.359 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.359 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.359 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.359 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.359 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.360 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.360 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.360 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.360 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.360 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.360 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.360 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.360 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.360 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.360 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.360 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.360 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.361 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.361 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.361 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.361 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.361 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.361 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.361 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.361 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.361 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.361 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.361 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.361 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.361 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.362 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.362 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.362 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.362 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.362 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.362 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.362 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.362 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.362 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.362 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.362 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.362 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.362 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.363 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.363 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.363 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.363 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.363 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.363 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.363 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.363 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.363 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.363 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.363 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.363 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.363 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.364 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.364 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.364 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.364 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.364 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.364 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.364 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.364 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.364 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.364 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.364 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.364 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.365 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.365 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.365 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.365 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.365 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.365 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.365 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.365 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.365 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.365 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.365 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.365 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.365 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.366 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.366 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.366 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.366 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.366 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.366 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.366 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.366 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.366 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.366 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.366 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.366 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.367 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.367 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.367 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.367 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.367 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.367 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.367 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.367 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.367 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.367 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.367 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.367 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.367 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.368 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.368 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.368 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.368 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.368 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.368 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.368 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.368 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.368 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.368 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.368 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.368 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.369 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.369 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.369 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.369 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.369 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.369 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.369 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.369 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.369 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.369 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.369 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.369 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.369 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.370 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.370 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.370 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.370 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.370 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.370 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.370 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.370 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.370 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.370 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.370 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.370 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.370 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.371 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.371 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.371 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.371 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.371 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.371 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.371 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.371 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.371 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.371 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.371 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.371 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.372 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.372 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.372 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.372 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.372 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.372 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.372 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.372 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.372 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.372 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.372 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.372 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.373 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.373 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.373 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.373 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.373 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.373 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.373 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.373 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.373 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.373 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.373 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.373 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.373 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.374 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.374 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.374 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.374 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.374 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.374 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.374 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.374 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.374 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.374 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.374 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.374 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.374 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.375 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.375 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.375 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.375 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.375 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.375 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.375 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.375 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.375 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.375 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.375 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.375 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.375 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.376 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.376 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.376 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.376 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.376 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.376 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.376 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.376 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.376 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.376 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.376 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.376 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.376 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.377 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.377 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.377 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.377 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.377 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.377 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.377 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.377 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.377 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.377 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.377 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.377 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.377 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.377 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.378 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.378 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.378 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.378 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.378 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.378 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.378 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.378 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.378 INFO:tasks.workunit.client.0.smithi110.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-09-15T03:38:52.378 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( ++i )) 2024-09-15T03:38:52.378 INFO:tasks.workunit.client.0.smithi110.stderr:++ (( i < 100 )) 2024-09-15T03:38:52.378 INFO:tasks.workunit.client.0.smithi110.stderr:+ background= 2024-09-15T03:38:52.378 INFO:tasks.workunit.client.0.smithi110.stderr:+ '[' '' = b ']' 2024-09-15T03:38:52.379 INFO:tasks.workunit.client.0.smithi110.stderr:++ cat 2024-09-15T03:38:52.379 INFO:tasks.workunit.client.0.smithi110.stderr:+ a='BEGIN TRANSACTION; 2024-09-15T03:38:52.379 INFO:tasks.workunit.client.0.smithi110.stderr:CREATE TABLE IF NOT EXISTS rand(text BLOB NOT NULL); 2024-09-15T03:38:52.379 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.379 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.379 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.379 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.379 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.379 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.379 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.379 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.379 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.379 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.380 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.380 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.380 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.380 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.380 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.380 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.380 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.380 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.380 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.380 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.380 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.380 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.381 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.381 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.381 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.381 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.381 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.381 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.381 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.381 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.381 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.381 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.381 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.381 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.381 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.382 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.382 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.382 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.382 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.382 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.382 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.382 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.382 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.382 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.382 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.382 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.382 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.382 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.382 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.383 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.383 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.383 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.383 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.383 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.383 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.383 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.383 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.383 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.383 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.383 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.383 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.383 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.384 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.384 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.384 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.384 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.384 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.384 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.384 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.384 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.384 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.384 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.384 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.384 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.384 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.385 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.385 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.385 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.385 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.385 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.385 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.385 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.385 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.385 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.385 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.385 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.385 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.386 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.386 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.386 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.386 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.386 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.386 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.386 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.386 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.386 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.386 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.386 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.386 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.386 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.387 INFO:tasks.workunit.client.0.smithi110.stderr:COMMIT; 2024-09-15T03:38:52.387 INFO:tasks.workunit.client.0.smithi110.stderr:SELECT LENGTH(text) FROM rand; 2024-09-15T03:38:52.387 INFO:tasks.workunit.client.0.smithi110.stderr:DROP TABLE rand;' 2024-09-15T03:38:52.387 INFO:tasks.workunit.client.0.smithi110.stderr:+ printf %s 'BEGIN TRANSACTION; 2024-09-15T03:38:52.387 INFO:tasks.workunit.client.0.smithi110.stderr:CREATE TABLE IF NOT EXISTS rand(text BLOB NOT NULL); 2024-09-15T03:38:52.387 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.387 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.387 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.387 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.387 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.387 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.387 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.387 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.387 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.388 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.388 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.388 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.388 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.388 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.388 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.388 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.388 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.388 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.388 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.388 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.389 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.389 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.389 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.389 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.389 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.389 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.389 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.389 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.389 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.389 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.390 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.390 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.390 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.390 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.390 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.390 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.390 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.390 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.390 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.390 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.390 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.390 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.390 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.391 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.391 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.391 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.391 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.391 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.391 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.391 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.391 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.391 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.391 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.391 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.391 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.391 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.392 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.392 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.392 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.392 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.392 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.392 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.392 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.392 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.392 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.392 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.392 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.392 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.393 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.393 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.393 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.393 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.393 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.393 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.393 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.393 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.393 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.393 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.393 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.393 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.393 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.394 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.394 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.394 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.394 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.394 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.394 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.394 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.394 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.394 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.394 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.394 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.394 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.394 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.395 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.395 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.395 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.395 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.395 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.395 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.395 INFO:tasks.workunit.client.0.smithi110.stderr:COMMIT; 2024-09-15T03:38:52.395 INFO:tasks.workunit.client.0.smithi110.stderr:SELECT LENGTH(text) FROM rand; 2024-09-15T03:38:52.395 INFO:tasks.workunit.client.0.smithi110.stderr:DROP TABLE rand;' 2024-09-15T03:38:52.395 INFO:tasks.workunit.client.0.smithi110.stderr:BEGIN TRANSACTION; 2024-09-15T03:38:52.395 INFO:tasks.workunit.client.0.smithi110.stderr:CREATE TABLE IF NOT EXISTS rand(text BLOB NOT NULL); 2024-09-15T03:38:52.395 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.395 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.396 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.396 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.396 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.396 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.396 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.396 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.396 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.396 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.396 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.396 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.396 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.396 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.397 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.397 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.397 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.397 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.397 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.397 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.397 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.397 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.397 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.397 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.397 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.397 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.397 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.398 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.398 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.398 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.398 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.398 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.398 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.398 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.398 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.398 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.398 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.398 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.398 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.399 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.399 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.399 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.399 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.399 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.399 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.399 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.399 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.399 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.399 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.399 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.399 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.400 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.400 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.400 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.400 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.401 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.401 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.401 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.401 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.401 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.401 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.401 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.401 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.401 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.401 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.401 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.401 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.401 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.402 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.402 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.402 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.402 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.402 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.402 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.402 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.402 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.402 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.402 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.402 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.402 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.402 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.403 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.403 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.403 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.403 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.403 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.403 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.403 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.403 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.403 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.403 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.403 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.403 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.403 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.404 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.404 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.404 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.404 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.404 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.404 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-09-15T03:38:52.404 INFO:tasks.workunit.client.0.smithi110.stderr:COMMIT; 2024-09-15T03:38:52.404 INFO:tasks.workunit.client.0.smithi110.stderr:SELECT LENGTH(text) FROM rand; 2024-09-15T03:38:52.404 INFO:tasks.workunit.client.0.smithi110.stderr:DROP TABLE rand;+ '[' '' '!=' b ']' 2024-09-15T03:38:52.404 INFO:tasks.workunit.client.0.smithi110.stderr:+ wait 2024-09-15T03:38:52.404 INFO:tasks.workunit.client.0.smithi110.stderr:+ sqlite3 -cmd '.output /dev/null' -cmd '.load libcephsqlite.so' -cmd 'pragma journal_mode = PERSIST' -cmd '.open file:///cephsqlite:test_libcephsqlite.sh/baz.db?vfs=ceph' -cmd '.output stdout' 2024-09-15T03:38:52.686 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.686 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.686 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.686 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.686 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.686 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.687 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.687 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.687 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.687 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.687 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.688 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.688 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.688 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.688 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.688 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.688 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.688 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.688 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.688 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.688 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.688 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.688 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.689 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.689 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.689 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.689 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.689 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.689 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.689 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.689 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.689 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.690 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.690 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.690 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.690 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.690 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.690 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.690 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.690 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.690 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.691 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.691 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.691 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.691 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.691 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.691 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.691 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.691 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.691 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.691 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.691 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.691 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.691 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.692 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.692 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.692 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.692 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.692 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.692 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.692 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.692 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.692 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.693 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.693 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.693 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.693 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.693 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.693 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.693 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.693 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.693 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.693 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.693 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.693 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.693 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.694 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.694 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.694 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.694 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.694 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.694 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.694 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.694 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.694 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.695 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.695 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.695 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.695 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.695 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.695 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.695 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.695 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.695 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.696 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.696 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.696 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.696 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.696 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:52.696 INFO:tasks.workunit.client.0.smithi110.stdout:4096 2024-09-15T03:38:53.692 INFO:tasks.workunit.client.0.smithi110.stderr: 2024-09-15T03:38:53.692 INFO:tasks.workunit.client.0.smithi110.stderr:real 0m1.360s 2024-09-15T03:38:53.692 INFO:tasks.workunit.client.0.smithi110.stderr:user 0m0.148s 2024-09-15T03:38:53.692 INFO:tasks.workunit.client.0.smithi110.stderr:sys 0m0.104s 2024-09-15T03:38:53.692 INFO:tasks.workunit.client.0.smithi110.stderr:+ striper rm baz.db 2024-09-15T03:38:53.693 INFO:tasks.workunit.client.0.smithi110.stderr:+ rados --pool=cephsqlite --namespace=test_libcephsqlite.sh --striper rm baz.db 2024-09-15T03:38:53.747 INFO:tasks.workunit.client.0.smithi110.stderr:+ date 2024-09-15T03:38:53.748 INFO:tasks.workunit.client.0.smithi110.stdout:Sun 15 Sep 2024 03:38:53 AM UTC 2024-09-15T03:38:53.749 INFO:tasks.workunit.client.0.smithi110.stderr:+ sqlite b 2024-09-15T03:38:53.749 INFO:tasks.workunit.client.0.smithi110.stderr:+ background=b 2024-09-15T03:38:53.749 INFO:tasks.workunit.client.0.smithi110.stderr:+ '[' b = b ']' 2024-09-15T03:38:53.749 INFO:tasks.workunit.client.0.smithi110.stderr:+ shift 2024-09-15T03:38:53.749 INFO:tasks.workunit.client.0.smithi110.stderr:++ cat 2024-09-15T03:38:53.750 INFO:tasks.workunit.client.0.smithi110.stderr:+ a='CREATE TABLE foo (a BLOB); 2024-09-15T03:38:53.750 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO foo VALUES ("start"); 2024-09-15T03:38:53.750 INFO:tasks.workunit.client.0.smithi110.stderr:WITH RECURSIVE c(x) AS 2024-09-15T03:38:53.750 INFO:tasks.workunit.client.0.smithi110.stderr: ( 2024-09-15T03:38:53.750 INFO:tasks.workunit.client.0.smithi110.stderr: VALUES(1) 2024-09-15T03:38:53.751 INFO:tasks.workunit.client.0.smithi110.stderr: UNION ALL 2024-09-15T03:38:53.751 INFO:tasks.workunit.client.0.smithi110.stderr: SELECT x+1 2024-09-15T03:38:53.751 INFO:tasks.workunit.client.0.smithi110.stderr: FROM c 2024-09-15T03:38:53.751 INFO:tasks.workunit.client.0.smithi110.stderr: ) 2024-09-15T03:38:53.751 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO foo (a) 2024-09-15T03:38:53.751 INFO:tasks.workunit.client.0.smithi110.stderr: SELECT RANDOMBLOB(1<<20) 2024-09-15T03:38:53.751 INFO:tasks.workunit.client.0.smithi110.stderr: FROM c 2024-09-15T03:38:53.751 INFO:tasks.workunit.client.0.smithi110.stderr: LIMIT (1<<20);' 2024-09-15T03:38:53.751 INFO:tasks.workunit.client.0.smithi110.stderr:+ printf %s 'CREATE TABLE foo (a BLOB); 2024-09-15T03:38:53.751 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO foo VALUES ("start"); 2024-09-15T03:38:53.751 INFO:tasks.workunit.client.0.smithi110.stderr:WITH RECURSIVE c(x) AS 2024-09-15T03:38:53.752 INFO:tasks.workunit.client.0.smithi110.stderr: ( 2024-09-15T03:38:53.752 INFO:tasks.workunit.client.0.smithi110.stderr: VALUES(1) 2024-09-15T03:38:53.752 INFO:tasks.workunit.client.0.smithi110.stderr: UNION ALL 2024-09-15T03:38:53.752 INFO:tasks.workunit.client.0.smithi110.stderr: SELECT x+1 2024-09-15T03:38:53.752 INFO:tasks.workunit.client.0.smithi110.stderr: FROM c 2024-09-15T03:38:53.752 INFO:tasks.workunit.client.0.smithi110.stderr: ) 2024-09-15T03:38:53.752 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO foo (a) 2024-09-15T03:38:53.752 INFO:tasks.workunit.client.0.smithi110.stderr: SELECT RANDOMBLOB(1<<20) 2024-09-15T03:38:53.752 INFO:tasks.workunit.client.0.smithi110.stderr: FROM c 2024-09-15T03:38:53.752 INFO:tasks.workunit.client.0.smithi110.stderr: LIMIT (1<<20);' 2024-09-15T03:38:53.752 INFO:tasks.workunit.client.0.smithi110.stderr:CREATE TABLE foo (a BLOB); 2024-09-15T03:38:53.753 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO foo VALUES ("start"); 2024-09-15T03:38:53.753 INFO:tasks.workunit.client.0.smithi110.stderr:WITH RECURSIVE c(x) AS 2024-09-15T03:38:53.753 INFO:tasks.workunit.client.0.smithi110.stderr: ( 2024-09-15T03:38:53.753 INFO:tasks.workunit.client.0.smithi110.stderr: VALUES(1) 2024-09-15T03:38:53.753 INFO:tasks.workunit.client.0.smithi110.stderr: UNION ALL 2024-09-15T03:38:53.753 INFO:tasks.workunit.client.0.smithi110.stderr: SELECT x+1 2024-09-15T03:38:53.753 INFO:tasks.workunit.client.0.smithi110.stderr: FROM c 2024-09-15T03:38:53.753 INFO:tasks.workunit.client.0.smithi110.stderr: ) 2024-09-15T03:38:53.753 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO foo (a) 2024-09-15T03:38:53.753 INFO:tasks.workunit.client.0.smithi110.stderr: SELECT RANDOMBLOB(1<<20) 2024-09-15T03:38:53.753 INFO:tasks.workunit.client.0.smithi110.stderr: FROM c 2024-09-15T03:38:53.753 INFO:tasks.workunit.client.0.smithi110.stderr: LIMIT (1<<20);+ '[' b '!=' b ']' 2024-09-15T03:38:53.754 INFO:tasks.workunit.client.0.smithi110.stderr:+ sleep 10 2024-09-15T03:38:53.754 INFO:tasks.workunit.client.0.smithi110.stderr:+ sqlite3 -cmd '.output /dev/null' -cmd '.load libcephsqlite.so' -cmd 'pragma journal_mode = PERSIST' -cmd '.open file:///cephsqlite:test_libcephsqlite.sh/baz.db?vfs=ceph' -cmd '.output stdout' 2024-09-15T03:39:03.751 INFO:tasks.workunit.client.0.smithi110.stdout:done 2024-09-15T03:39:03.751 INFO:tasks.workunit.client.0.smithi110.stderr:+ echo done 2024-09-15T03:39:03.752 INFO:tasks.workunit.client.0.smithi110.stderr:+ jobs -l 2024-09-15T03:39:03.752 INFO:tasks.workunit.client.0.smithi110.stdout:[1]+ 27314 Running sqlite3 -cmd '.output /dev/null' -cmd '.load libcephsqlite.so' -cmd 'pragma journal_mode = PERSIST' -cmd ".open file:///$pool:$ns/baz.db?vfs=ceph" -cmd '.output stdout' <<< "$a" & 2024-09-15T03:39:03.752 INFO:tasks.workunit.client.0.smithi110.stderr:++ jobs -p 2024-09-15T03:39:03.752 INFO:tasks.workunit.client.0.smithi110.stderr:+ kill -KILL -- 27314 2024-09-15T03:39:03.753 INFO:tasks.workunit.client.0.smithi110.stderr:+ date 2024-09-15T03:39:03.753 INFO:tasks.workunit.client.0.smithi110.stdout:Sun 15 Sep 2024 03:39:03 AM UTC 2024-09-15T03:39:03.753 INFO:tasks.workunit.client.0.smithi110.stderr:+ wait 2024-09-15T03:39:03.755 INFO:tasks.workunit.client.0.smithi110.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_libcephsqlite.sh: line 95: 27314 Killed sqlite3 -cmd '.output /dev/null' -cmd '.load libcephsqlite.so' -cmd 'pragma journal_mode = PERSIST' -cmd ".open file:///$pool:$ns/baz.db?vfs=ceph" -cmd '.output stdout' <<< "$a" 2024-09-15T03:39:03.755 INFO:tasks.workunit.client.0.smithi110.stderr:+ date 2024-09-15T03:39:03.756 INFO:tasks.workunit.client.0.smithi110.stdout:Sun 15 Sep 2024 03:39:03 AM UTC 2024-09-15T03:39:03.756 INFO:tasks.workunit.client.0.smithi110.stderr:++ sqlite 2024-09-15T03:39:03.757 INFO:tasks.workunit.client.0.smithi110.stderr:++ background= 2024-09-15T03:39:03.757 INFO:tasks.workunit.client.0.smithi110.stderr:++ '[' '' = b ']' 2024-09-15T03:39:03.757 INFO:tasks.workunit.client.0.smithi110.stderr:+++ cat 2024-09-15T03:39:03.758 INFO:tasks.workunit.client.0.smithi110.stderr:++ a='SELECT COUNT(*) FROM foo;' 2024-09-15T03:39:03.758 INFO:tasks.workunit.client.0.smithi110.stderr:++ printf %s 'SELECT COUNT(*) FROM foo;' 2024-09-15T03:39:03.758 INFO:tasks.workunit.client.0.smithi110.stderr:SELECT COUNT(*) FROM foo;++ '[' '' '!=' b ']' 2024-09-15T03:39:03.758 INFO:tasks.workunit.client.0.smithi110.stderr:++ wait 2024-09-15T03:39:03.758 INFO:tasks.workunit.client.0.smithi110.stderr:++ sqlite3 -cmd '.output /dev/null' -cmd '.load libcephsqlite.so' -cmd 'pragma journal_mode = PERSIST' -cmd '.open file:///cephsqlite:test_libcephsqlite.sh/baz.db?vfs=ceph' -cmd '.output stdout' 2024-09-15T03:39:32.488 INFO:tasks.workunit.client.0.smithi110.stderr:+ n=1 2024-09-15T03:39:32.489 INFO:tasks.workunit.client.0.smithi110.stderr:+ '[' 1 -eq 1 ']' 2024-09-15T03:39:32.489 INFO:tasks.workunit.client.0.smithi110.stderr:+ striper rm baz.db 2024-09-15T03:39:32.489 INFO:tasks.workunit.client.0.smithi110.stderr:+ rados --pool=cephsqlite --namespace=test_libcephsqlite.sh --striper rm baz.db 2024-09-15T03:39:32.553 INFO:tasks.workunit.client.0.smithi110.stderr:+ date 2024-09-15T03:39:32.555 INFO:tasks.workunit.client.0.smithi110.stdout:Sun 15 Sep 2024 03:39:32 AM UTC 2024-09-15T03:39:32.556 INFO:tasks.workunit.client.0.smithi110.stderr:+ sqlite b 2024-09-15T03:39:32.556 INFO:tasks.workunit.client.0.smithi110.stderr:+ background=b 2024-09-15T03:39:32.556 INFO:tasks.workunit.client.0.smithi110.stderr:+ '[' b = b ']' 2024-09-15T03:39:32.556 INFO:tasks.workunit.client.0.smithi110.stderr:+ shift 2024-09-15T03:39:32.556 INFO:tasks.workunit.client.0.smithi110.stderr:++ cat 2024-09-15T03:39:32.557 INFO:tasks.workunit.client.0.smithi110.stderr:+ a='CREATE TABLE foo (a BLOB); 2024-09-15T03:39:32.557 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO foo VALUES ("start"); 2024-09-15T03:39:32.557 INFO:tasks.workunit.client.0.smithi110.stderr:WITH RECURSIVE c(x) AS 2024-09-15T03:39:32.557 INFO:tasks.workunit.client.0.smithi110.stderr: ( 2024-09-15T03:39:32.557 INFO:tasks.workunit.client.0.smithi110.stderr: VALUES(1) 2024-09-15T03:39:32.557 INFO:tasks.workunit.client.0.smithi110.stderr: UNION ALL 2024-09-15T03:39:32.557 INFO:tasks.workunit.client.0.smithi110.stderr: SELECT x+1 2024-09-15T03:39:32.557 INFO:tasks.workunit.client.0.smithi110.stderr: FROM c 2024-09-15T03:39:32.558 INFO:tasks.workunit.client.0.smithi110.stderr: ) 2024-09-15T03:39:32.558 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO foo (a) 2024-09-15T03:39:32.558 INFO:tasks.workunit.client.0.smithi110.stderr: SELECT RANDOMBLOB(1<<20) 2024-09-15T03:39:32.558 INFO:tasks.workunit.client.0.smithi110.stderr: FROM c 2024-09-15T03:39:32.558 INFO:tasks.workunit.client.0.smithi110.stderr: LIMIT (1<<20);' 2024-09-15T03:39:32.558 INFO:tasks.workunit.client.0.smithi110.stderr:+ printf %s 'CREATE TABLE foo (a BLOB); 2024-09-15T03:39:32.558 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO foo VALUES ("start"); 2024-09-15T03:39:32.558 INFO:tasks.workunit.client.0.smithi110.stderr:WITH RECURSIVE c(x) AS 2024-09-15T03:39:32.558 INFO:tasks.workunit.client.0.smithi110.stderr: ( 2024-09-15T03:39:32.558 INFO:tasks.workunit.client.0.smithi110.stderr: VALUES(1) 2024-09-15T03:39:32.558 INFO:tasks.workunit.client.0.smithi110.stderr: UNION ALL 2024-09-15T03:39:32.559 INFO:tasks.workunit.client.0.smithi110.stderr: SELECT x+1 2024-09-15T03:39:32.559 INFO:tasks.workunit.client.0.smithi110.stderr: FROM c 2024-09-15T03:39:32.559 INFO:tasks.workunit.client.0.smithi110.stderr: ) 2024-09-15T03:39:32.559 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO foo (a) 2024-09-15T03:39:32.559 INFO:tasks.workunit.client.0.smithi110.stderr: SELECT RANDOMBLOB(1<<20) 2024-09-15T03:39:32.559 INFO:tasks.workunit.client.0.smithi110.stderr: FROM c 2024-09-15T03:39:32.559 INFO:tasks.workunit.client.0.smithi110.stderr: LIMIT (1<<20);' 2024-09-15T03:39:32.559 INFO:tasks.workunit.client.0.smithi110.stderr:CREATE TABLE foo (a BLOB); 2024-09-15T03:39:32.559 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO foo VALUES ("start"); 2024-09-15T03:39:32.559 INFO:tasks.workunit.client.0.smithi110.stderr:WITH RECURSIVE c(x) AS 2024-09-15T03:39:32.559 INFO:tasks.workunit.client.0.smithi110.stderr: ( 2024-09-15T03:39:32.559 INFO:tasks.workunit.client.0.smithi110.stderr: VALUES(1) 2024-09-15T03:39:32.560 INFO:tasks.workunit.client.0.smithi110.stderr: UNION ALL 2024-09-15T03:39:32.560 INFO:tasks.workunit.client.0.smithi110.stderr: SELECT x+1 2024-09-15T03:39:32.560 INFO:tasks.workunit.client.0.smithi110.stderr: FROM c 2024-09-15T03:39:32.560 INFO:tasks.workunit.client.0.smithi110.stderr: ) 2024-09-15T03:39:32.560 INFO:tasks.workunit.client.0.smithi110.stderr:INSERT INTO foo (a) 2024-09-15T03:39:32.560 INFO:tasks.workunit.client.0.smithi110.stderr: SELECT RANDOMBLOB(1<<20) 2024-09-15T03:39:32.560 INFO:tasks.workunit.client.0.smithi110.stderr: FROM c 2024-09-15T03:39:32.560 INFO:tasks.workunit.client.0.smithi110.stderr: LIMIT (1<<20);+ '[' b '!=' b ']' 2024-09-15T03:39:32.560 INFO:tasks.workunit.client.0.smithi110.stderr:+ sleep 20 2024-09-15T03:39:32.560 INFO:tasks.workunit.client.0.smithi110.stderr:+ sqlite3 -cmd '.output /dev/null' -cmd '.load libcephsqlite.so' -cmd 'pragma journal_mode = PERSIST' -cmd '.open file:///cephsqlite:test_libcephsqlite.sh/baz.db?vfs=ceph' -cmd '.output stdout' 2024-09-15T03:39:52.559 INFO:tasks.workunit.client.0.smithi110.stderr:+ jobs -l 2024-09-15T03:39:52.559 INFO:tasks.workunit.client.0.smithi110.stdout:[1]+ 27382 Running sqlite3 -cmd '.output /dev/null' -cmd '.load libcephsqlite.so' -cmd 'pragma journal_mode = PERSIST' -cmd ".open file:///$pool:$ns/baz.db?vfs=ceph" -cmd '.output stdout' <<< "$a" & 2024-09-15T03:39:52.560 INFO:tasks.workunit.client.0.smithi110.stderr:++ jobs -p 2024-09-15T03:39:52.560 INFO:tasks.workunit.client.0.smithi110.stderr:+ kill -STOP -- 27382 2024-09-15T03:39:52.560 INFO:tasks.workunit.client.0.smithi110.stderr:+ sleep 45 2024-09-15T03:40:37.561 INFO:tasks.workunit.client.0.smithi110.stderr:+ date 2024-09-15T03:40:37.562 INFO:tasks.workunit.client.0.smithi110.stdout:Sun 15 Sep 2024 03:40:37 AM UTC 2024-09-15T03:40:37.562 INFO:tasks.workunit.client.0.smithi110.stderr:++ jobs -p 2024-09-15T03:40:37.563 INFO:tasks.workunit.client.0.smithi110.stderr:+ kill -CONT -- 27382 2024-09-15T03:40:37.563 INFO:tasks.workunit.client.0.smithi110.stderr:+ sleep 10 2024-09-15T03:40:37.591 INFO:tasks.workunit.client.0.smithi110.stderr:Error: near line 3: disk I/O error 2024-09-15T03:40:47.565 INFO:tasks.workunit.client.0.smithi110.stderr:+ date 2024-09-15T03:40:47.566 INFO:tasks.workunit.client.0.smithi110.stdout:Sun 15 Sep 2024 03:40:47 AM UTC 2024-09-15T03:40:47.567 INFO:tasks.workunit.client.0.smithi110.stderr:+ wait 2024-09-15T03:40:47.567 INFO:tasks.workunit.client.0.smithi110.stderr:+ date 2024-09-15T03:40:47.568 INFO:tasks.workunit.client.0.smithi110.stdout:Sun 15 Sep 2024 03:40:47 AM UTC 2024-09-15T03:40:47.569 INFO:tasks.workunit.client.0.smithi110.stderr:++ sqlite 2024-09-15T03:40:47.569 INFO:tasks.workunit.client.0.smithi110.stderr:++ background= 2024-09-15T03:40:47.569 INFO:tasks.workunit.client.0.smithi110.stderr:++ '[' '' = b ']' 2024-09-15T03:40:47.570 INFO:tasks.workunit.client.0.smithi110.stderr:+++ cat 2024-09-15T03:40:47.571 INFO:tasks.workunit.client.0.smithi110.stderr:++ a='SELECT COUNT(*) FROM foo;' 2024-09-15T03:40:47.571 INFO:tasks.workunit.client.0.smithi110.stderr:++ printf %s 'SELECT COUNT(*) FROM foo;' 2024-09-15T03:40:47.572 INFO:tasks.workunit.client.0.smithi110.stderr:SELECT COUNT(*) FROM foo;++ '[' '' '!=' b ']' 2024-09-15T03:40:47.572 INFO:tasks.workunit.client.0.smithi110.stderr:++ wait 2024-09-15T03:40:47.572 INFO:tasks.workunit.client.0.smithi110.stderr:++ sqlite3 -cmd '.output /dev/null' -cmd '.load libcephsqlite.so' -cmd 'pragma journal_mode = PERSIST' -cmd '.open file:///cephsqlite:test_libcephsqlite.sh/baz.db?vfs=ceph' -cmd '.output stdout' 2024-09-15T03:40:48.368 INFO:tasks.workunit.client.0.smithi110.stderr:+ n=1 2024-09-15T03:40:48.369 INFO:tasks.workunit.client.0.smithi110.stderr:+ '[' 1 -eq 1 ']' 2024-09-15T03:40:48.369 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-15T03:40:48.369 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-15T03:40:48.424 INFO:tasks.workunit:Stopping ['rados/test_libcephsqlite.sh cephsqlite'] on client.0... 2024-09-15T03:40:48.424 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-15T03:40:48.689 DEBUG:teuthology.parallel:result is None 2024-09-15T03:40:48.689 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-15T03:40:48.705 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-15T03:40:48.705 DEBUG:teuthology.orchestra.run.smithi110:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-09-15T03:40:48.761 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-09-15T03:40:48.761 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-15T03:40:48.771 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2024-09-15T03:40:48.771 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T03:40:48.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:48.875+0000 7fbf42340700 1 Processor -- start 2024-09-15T03:40:48.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:48.875+0000 7fbf42340700 1 -- start start 2024-09-15T03:40:48.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:48.875+0000 7fbf42340700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fbf3c0af940 0x7fbf3c0a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:48.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:48.879+0000 7fbf42340700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fbf3c13d080 0x7fbf3c13d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:48.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:48.879+0000 7fbf42340700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf3c0a53b0 con 0x7fbf3c0b00c0 2024-09-15T03:40:48.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:48.879+0000 7fbf42340700 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf3c0a6310 con 0x7fbf3c13d080 2024-09-15T03:40:48.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:48.879+0000 7fbf42340700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3c0afd10 con 0x7fbf3c0af940 2024-09-15T03:40:48.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:48.879+0000 7fbf408dd700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fbf3c13d080 0x7fbf3c13d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:48.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:48.879+0000 7fbf3bfff700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fbf3c0af940 0x7fbf3c0a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:48.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:48.879+0000 7fbf408dd700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fbf3c13d080 0x7fbf3c13d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51020/0 (socket says 172.21.15.110:51020) 2024-09-15T03:40:48.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:48.879+0000 7fbf3bfff700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fbf3c0af940 0x7fbf3c0a4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3301/0 says I am v2:172.21.15.110:42928/0 (socket says 172.21.15.110:42928) 2024-09-15T03:40:48.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:48.879+0000 7fbf408dd700 1 -- 172.21.15.110:0/3047145651 learned_addr learned my addr 172.21.15.110:0/3047145651 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:40:48.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:48.879+0000 7fbf408dd700 1 -- 172.21.15.110:0/3047145651 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fbf3c0af940 msgr2=0x7fbf3c0a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:40:48.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:48.879+0000 7fbf408dd700 1 --2- 172.21.15.110:0/3047145651 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fbf3c0af940 0x7fbf3c0a4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:48.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:48.879+0000 7fbf408dd700 1 -- 172.21.15.110:0/3047145651 >> v1:172.21.15.177:6789/0 conn(0x7fbf3c0b00c0 legacy=0x7fbf3c068810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:48.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:48.879+0000 7fbf408dd700 1 -- 172.21.15.110:0/3047145651 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf3c13ee60 con 0x7fbf3c13d080 2024-09-15T03:40:48.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:48.879+0000 7fbf3b7fe700 1 -- 172.21.15.110:0/3047145651 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1389006636 0 0) 0x7fbf24001110 con 0x7fbf3c0b00c0 2024-09-15T03:40:48.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:48.879+0000 7fbf3bfff700 1 --2- 172.21.15.110:0/3047145651 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fbf3c0af940 0x7fbf3c0a4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T03:40:48.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:48.879+0000 7fbf408dd700 1 --2- 172.21.15.110:0/3047145651 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf3c13d080 0x7fbf3c13d4f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c13f050 tx=0x7fbf34009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a181f8ea32111d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:48.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:48.879+0000 7fbf3b7fe700 1 -- 172.21.15.110:0/3047145651 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf3400e070 con 0x7fbf3c13d080 2024-09-15T03:40:48.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:48.879+0000 7fbf3b7fe700 1 -- 172.21.15.110:0/3047145651 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf3400ba20 con 0x7fbf3c13d080 2024-09-15T03:40:48.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:48.879+0000 7fbf3b7fe700 1 -- 172.21.15.110:0/3047145651 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf3400bde0 con 0x7fbf3c13d080 2024-09-15T03:40:48.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:48.879+0000 7fbf42340700 1 -- 172.21.15.110:0/3047145651 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf3c13d080 msgr2=0x7fbf3c13d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:48.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:48.879+0000 7fbf42340700 1 --2- 172.21.15.110:0/3047145651 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf3c13d080 0x7fbf3c13d4f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c13f050 tx=0x7fbf34009840 comp rx=0 tx=0).stop 2024-09-15T03:40:48.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:48.879+0000 7fbf42340700 1 -- 172.21.15.110:0/3047145651 shutdown_connections 2024-09-15T03:40:48.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:48.879+0000 7fbf42340700 1 --2- 172.21.15.110:0/3047145651 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf3c13d080 0x7fbf3c13d4f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:48.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:48.879+0000 7fbf42340700 1 --2- 172.21.15.110:0/3047145651 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fbf3c0af940 0x7fbf3c0a4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:48.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:48.879+0000 7fbf42340700 1 -- 172.21.15.110:0/3047145651 >> 172.21.15.110:0/3047145651 conn(0x7fbf3c0b1810 msgr2=0x7fbf3c0b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:40:48.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:48.879+0000 7fbf42340700 1 -- 172.21.15.110:0/3047145651 shutdown_connections 2024-09-15T03:40:48.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:48.879+0000 7fbf42340700 1 -- 172.21.15.110:0/3047145651 wait complete. 2024-09-15T03:40:48.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:48.883+0000 7fbf42340700 1 Processor -- start 2024-09-15T03:40:48.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:48.883+0000 7fbf42340700 1 -- start start 2024-09-15T03:40:48.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:48.883+0000 7fbf42340700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fbf3c0af940 0x7fbf3c06e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:48.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:48.883+0000 7fbf42340700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf3c0b00c0 0x7fbf3c06d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:48.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:48.883+0000 7fbf42340700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf3c13d080 0x7fbf3c14d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:48.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:48.883+0000 7fbf42340700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3c0aafb0 con 0x7fbf3c13d080 2024-09-15T03:40:48.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:48.883+0000 7fbf42340700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3c06dd20 con 0x7fbf3c0b00c0 2024-09-15T03:40:48.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:48.883+0000 7fbf42340700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3c0ab240 con 0x7fbf3c0af940 2024-09-15T03:40:48.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:48.883+0000 7fbf3bfff700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fbf3c0af940 0x7fbf3c06e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:48.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:48.883+0000 7fbf408dd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf3c13d080 0x7fbf3c14d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:48.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:48.883+0000 7fbf3bfff700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fbf3c0af940 0x7fbf3c06e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3301/0 says I am v2:172.21.15.110:42944/0 (socket says 172.21.15.110:42944) 2024-09-15T03:40:48.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:48.883+0000 7fbf3bfff700 1 -- 172.21.15.110:0/3028470489 learned_addr learned my addr 172.21.15.110:0/3028470489 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:40:48.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:48.883+0000 7fbf408dd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf3c13d080 0x7fbf3c14d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51022/0 (socket says 172.21.15.110:51022) 2024-09-15T03:40:48.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:48.883+0000 7fbf33fff700 1 --2- 172.21.15.110:0/3028470489 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf3c0b00c0 0x7fbf3c06d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:48.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:48.883+0000 7fbf3bfff700 1 -- 172.21.15.110:0/3028470489 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf3c0b00c0 msgr2=0x7fbf3c06d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:48.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:48.883+0000 7fbf3bfff700 1 --2- 172.21.15.110:0/3028470489 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf3c0b00c0 0x7fbf3c06d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:48.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:48.883+0000 7fbf3bfff700 1 -- 172.21.15.110:0/3028470489 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf3c13d080 msgr2=0x7fbf3c14d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:48.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:48.883+0000 7fbf3bfff700 1 --2- 172.21.15.110:0/3028470489 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf3c13d080 0x7fbf3c14d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:48.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:48.883+0000 7fbf3bfff700 1 -- 172.21.15.110:0/3028470489 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf2c009420 con 0x7fbf3c0af940 2024-09-15T03:40:48.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:48.883+0000 7fbf408dd700 1 --2- 172.21.15.110:0/3028470489 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf3c13d080 0x7fbf3c14d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T03:40:48.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:48.883+0000 7fbf3bfff700 1 --2- 172.21.15.110:0/3028470489 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fbf3c0af940 0x7fbf3c06e2e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c00ec80 tx=0x7fbf2c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:48.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:48.883+0000 7fbf38ff9700 1 -- 172.21.15.110:0/3028470489 <== mon.2 v2:172.21.15.110:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2c00c920 con 0x7fbf3c0af940 2024-09-15T03:40:48.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:48.883+0000 7fbf38ff9700 1 -- 172.21.15.110:0/3028470489 <== mon.2 v2:172.21.15.110:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf2c00ca80 con 0x7fbf3c0af940 2024-09-15T03:40:48.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:48.883+0000 7fbf42340700 1 -- 172.21.15.110:0/3028470489 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf34009ae0 con 0x7fbf3c0af940 2024-09-15T03:40:48.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:48.883+0000 7fbf38ff9700 1 -- 172.21.15.110:0/3028470489 <== mon.2 v2:172.21.15.110:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2c00ce00 con 0x7fbf3c0af940 2024-09-15T03:40:48.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:48.883+0000 7fbf42340700 1 -- 172.21.15.110:0/3028470489 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf3c13ac00 con 0x7fbf3c0af940 2024-09-15T03:40:48.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:48.887+0000 7fbf42340700 1 -- 172.21.15.110:0/3028470489 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf3c13aee0 con 0x7fbf3c0af940 2024-09-15T03:40:48.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:48.887+0000 7fbf38ff9700 1 -- 172.21.15.110:0/3028470489 <== mon.2 v2:172.21.15.110:3301/0 4 ==== mgrmap(e 5) v1 ==== 85847+0+0 (secure 0 0 0) 0x7fbf2c013680 con 0x7fbf3c0af940 2024-09-15T03:40:48.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:48.887+0000 7fbf38ff9700 1 --2- 172.21.15.110:0/3028470489 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fbf1c067a80 0x7fbf1c069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:48.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:48.887+0000 7fbf38ff9700 1 -- 172.21.15.110:0/3028470489 <== mon.2 v2:172.21.15.110:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5641+0+0 (secure 0 0 0) 0x7fbf2c085ad0 con 0x7fbf3c0af940 2024-09-15T03:40:48.891 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:48.887+0000 7fbf33fff700 1 --2- 172.21.15.110:0/3028470489 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fbf1c067a80 0x7fbf1c069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:48.892 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:48.887+0000 7fbf33fff700 1 --2- 172.21.15.110:0/3028470489 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fbf1c067a80 0x7fbf1c069f30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c143f90 tx=0x7fbf24007ba0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:48.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:48.895+0000 7fbf38ff9700 1 -- 172.21.15.110:0/3028470489 <== mon.2 v2:172.21.15.110:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fbf2c052b70 con 0x7fbf3c0af940 2024-09-15T03:40:49.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.063+0000 7fbf42340700 1 -- 172.21.15.110:0/3028470489 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbf3c0721b0 con 0x7fbf3c0af940 2024-09-15T03:40:49.069 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.067+0000 7fbf38ff9700 1 -- 172.21.15.110:0/3028470489 <== mon.2 v2:172.21.15.110:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+15365 (secure 0 0 0) 0x7fbf2c017090 con 0x7fbf3c0af940 2024-09-15T03:40:49.070 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-15T03:40:49.070 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":24,"fsid":"d7bdbc6e-e9f0-40e5-921a-f7c68fa2d8c1","created":"2024-09-15T03:36:28.375926+0000","modified":"2024-09-15T03:40:48.246453+0000","last_up_change":"2024-09-15T03:36:37.618040+0000","last_in_change":"2024-09-15T03:36:30.385381+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T03:36:36.657583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-15T03:36:39.048914+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephsqlite","create_time":"2024-09-15T03:36:47.199176+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{}}],"osds":[{"osd":0,"uuid":"09d93c1d-8ec1-42e8-8b59-5e20958830d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":22352},{"type":"v1","addr":"172.21.15.110:6801","nonce":22352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":22352},{"type":"v1","addr":"172.21.15.110:6803","nonce":22352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":22352},{"type":"v1","addr":"172.21.15.110:6807","nonce":22352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":22352},{"type":"v1","addr":"172.21.15.110:6805","nonce":22352}]},"public_addr":"172.21.15.110:6801/22352","cluster_addr":"172.21.15.110:6803/22352","heartbeat_back_addr":"172.21.15.110:6807/22352","heartbeat_front_addr":"172.21.15.110:6805/22352","state":["exists","up"]},{"osd":1,"uuid":"378acca0-6784-42c1-a6f8-0ba39a287db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":22354},{"type":"v1","addr":"172.21.15.110:6809","nonce":22354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":22354},{"type":"v1","addr":"172.21.15.110:6811","nonce":22354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":22354},{"type":"v1","addr":"172.21.15.110:6815","nonce":22354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":22354},{"type":"v1","addr":"172.21.15.110:6813","nonce":22354}]},"public_addr":"172.21.15.110:6809/22354","cluster_addr":"172.21.15.110:6811/22354","heartbeat_back_addr":"172.21.15.110:6815/22354","heartbeat_front_addr":"172.21.15.110:6813/22354","state":["exists","up"]},{"osd":2,"uuid":"b1c2c060-985a-429c-bf05-ccfd5fdf3d8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":22356},{"type":"v1","addr":"172.21.15.110:6824","nonce":22356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":22356},{"type":"v1","addr":"172.21.15.110:6827","nonce":22356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":22356},{"type":"v1","addr":"172.21.15.110:6831","nonce":22356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":22356},{"type":"v1","addr":"172.21.15.110:6829","nonce":22356}]},"public_addr":"172.21.15.110:6824/22356","cluster_addr":"172.21.15.110:6827/22356","heartbeat_back_addr":"172.21.15.110:6831/22356","heartbeat_front_addr":"172.21.15.110:6829/22356","state":["exists","up"]},{"osd":3,"uuid":"aca55104-d76b-4719-8837-252c2e55d0cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":22355},{"type":"v1","addr":"172.21.15.110:6817","nonce":22355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":22355},{"type":"v1","addr":"172.21.15.110:6819","nonce":22355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6823","nonce":22355},{"type":"v1","addr":"172.21.15.110:6825","nonce":22355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":22355},{"type":"v1","addr":"172.21.15.110:6821","nonce":22355}]},"public_addr":"172.21.15.110:6817/22355","cluster_addr":"172.21.15.110:6819/22355","heartbeat_back_addr":"172.21.15.110:6825/22355","heartbeat_front_addr":"172.21.15.110:6821/22355","state":["exists","up"]},{"osd":4,"uuid":"7a87be3c-b6ff-43a4-8672-39d3006a0e2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":22425},{"type":"v1","addr":"172.21.15.177:6824","nonce":22425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":22425},{"type":"v1","addr":"172.21.15.177:6827","nonce":22425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":22425},{"type":"v1","addr":"172.21.15.177:6831","nonce":22425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":22425},{"type":"v1","addr":"172.21.15.177:6829","nonce":22425}]},"public_addr":"172.21.15.177:6824/22425","cluster_addr":"172.21.15.177:6827/22425","heartbeat_back_addr":"172.21.15.177:6831/22425","heartbeat_front_addr":"172.21.15.177:6829/22425","state":["exists","up"]},{"osd":5,"uuid":"a8f3a63d-45af-41fe-be33-2339d7b9ed96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":22424},{"type":"v1","addr":"172.21.15.177:6804","nonce":22424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":22424},{"type":"v1","addr":"172.21.15.177:6808","nonce":22424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":22424},{"type":"v1","addr":"172.21.15.177:6815","nonce":22424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":22424},{"type":"v1","addr":"172.21.15.177:6812","nonce":22424}]},"public_addr":"172.21.15.177:6804/22424","cluster_addr":"172.21.15.177:6808/22424","heartbeat_back_addr":"172.21.15.177:6815/22424","heartbeat_front_addr":"172.21.15.177:6812/22424","state":["exists","up"]},{"osd":6,"uuid":"a61b52dd-273d-47b4-bf92-0610cce86e1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":22428},{"type":"v1","addr":"172.21.15.177:6817","nonce":22428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":22428},{"type":"v1","addr":"172.21.15.177:6819","nonce":22428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":22428},{"type":"v1","addr":"172.21.15.177:6825","nonce":22428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":22428},{"type":"v1","addr":"172.21.15.177:6821","nonce":22428}]},"public_addr":"172.21.15.177:6817/22428","cluster_addr":"172.21.15.177:6819/22428","heartbeat_back_addr":"172.21.15.177:6825/22428","heartbeat_front_addr":"172.21.15.177:6821/22428","state":["exists","up"]},{"osd":7,"uuid":"1a7d5a9f-6b41-47e1-b0ef-45f4d638fd65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":22423},{"type":"v1","addr":"172.21.15.177:6801","nonce":22423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6803","nonce":22423},{"type":"v1","addr":"172.21.15.177:6805","nonce":22423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6811","nonce":22423},{"type":"v1","addr":"172.21.15.177:6813","nonce":22423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6807","nonce":22423},{"type":"v1","addr":"172.21.15.177:6809","nonce":22423}]},"public_addr":"172.21.15.177:6801/22423","cluster_addr":"172.21.15.177:6805/22423","heartbeat_back_addr":"172.21.15.177:6813/22423","heartbeat_front_addr":"172.21.15.177:6809/22423","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:36:35.106295+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:36:35.495899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:36:35.902086+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:36:35.652162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:36:35.645490+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:36:35.658420+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:36:35.622386+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:36:35.604615+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.110:0/4006587460":"2024-09-15T04:40:47.640898+0000","172.21.15.110:0/3981642940":"2024-09-15T04:39:31.839305+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T03:40:49.075 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.071+0000 7fbf42340700 1 -- 172.21.15.110:0/3028470489 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fbf1c067a80 msgr2=0x7fbf1c069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:49.075 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.071+0000 7fbf42340700 1 --2- 172.21.15.110:0/3028470489 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fbf1c067a80 0x7fbf1c069f30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c143f90 tx=0x7fbf24007ba0 comp rx=0 tx=0).stop 2024-09-15T03:40:49.075 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.071+0000 7fbf42340700 1 -- 172.21.15.110:0/3028470489 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fbf3c0af940 msgr2=0x7fbf3c06e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:49.075 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.071+0000 7fbf42340700 1 --2- 172.21.15.110:0/3028470489 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fbf3c0af940 0x7fbf3c06e2e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c00ec80 tx=0x7fbf2c0091a0 comp rx=0 tx=0).stop 2024-09-15T03:40:49.075 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.071+0000 7fbf42340700 1 -- 172.21.15.110:0/3028470489 shutdown_connections 2024-09-15T03:40:49.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.071+0000 7fbf42340700 1 --2- 172.21.15.110:0/3028470489 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fbf1c067a80 0x7fbf1c069f30 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:49.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.071+0000 7fbf42340700 1 --2- 172.21.15.110:0/3028470489 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf3c13d080 0x7fbf3c14d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:49.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.071+0000 7fbf42340700 1 --2- 172.21.15.110:0/3028470489 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf3c0b00c0 0x7fbf3c06d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:49.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.071+0000 7fbf42340700 1 --2- 172.21.15.110:0/3028470489 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fbf3c0af940 0x7fbf3c06e2e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:49.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.071+0000 7fbf42340700 1 -- 172.21.15.110:0/3028470489 >> 172.21.15.110:0/3028470489 conn(0x7fbf3c0b1810 msgr2=0x7fbf3c0a5550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:40:49.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.071+0000 7fbf42340700 1 -- 172.21.15.110:0/3028470489 shutdown_connections 2024-09-15T03:40:49.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.071+0000 7fbf42340700 1 -- 172.21.15.110:0/3028470489 wait complete. 2024-09-15T03:40:49.091 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-15T03:40:49.091 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T03:40:49.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.191+0000 7f340ffff700 1 Processor -- start 2024-09-15T03:40:49.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.191+0000 7f340ffff700 1 -- start start 2024-09-15T03:40:49.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.191+0000 7f340ffff700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f34100b3030 0x7f34100adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:49.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.191+0000 7f340ffff700 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f341006eca0 con 0x7f341006b860 2024-09-15T03:40:49.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.191+0000 7f340ffff700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34100b3400 con 0x7f341006bc90 2024-09-15T03:40:49.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.191+0000 7f340ffff700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34100ad890 con 0x7f34100b3030 2024-09-15T03:40:49.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.191+0000 7f340e7fc700 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f341006b860 0x7f341006de80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:32852/0 (socket says 172.21.15.110:32852) 2024-09-15T03:40:49.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.191+0000 7f340e7fc700 1 -- 172.21.15.110:0/1320655650 learned_addr learned my addr 172.21.15.110:0/1320655650 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-15T03:40:49.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.191+0000 7f340effd700 1 --2- 172.21.15.110:0/1320655650 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f34100b3030 0x7f34100adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:49.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.191+0000 7f340dffb700 1 -- 172.21.15.110:0/1320655650 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2582744586 0 0) 0x7f341006eca0 con 0x7f341006b860 2024-09-15T03:40:49.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.191+0000 7f340dffb700 1 -- 172.21.15.110:0/1320655650 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33f8003570 con 0x7f341006b860 2024-09-15T03:40:49.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.191+0000 7f340dffb700 1 -- 172.21.15.110:0/1320655650 <== mon.0 v1:172.21.15.110:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2161977518 0 0) 0x7f33f8003570 con 0x7f341006b860 2024-09-15T03:40:49.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.191+0000 7f340dffb700 1 -- 172.21.15.110:0/1320655650 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f34100b3030 msgr2=0x7f34100adfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:40:49.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.191+0000 7f340dffb700 1 --2- 172.21.15.110:0/1320655650 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f34100b3030 0x7f34100adfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:49.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.191+0000 7f340dffb700 1 -- 172.21.15.110:0/1320655650 >> v1:172.21.15.177:6789/0 conn(0x7f341006bc90 legacy=0x7f341006e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:49.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.195+0000 7f340dffb700 1 -- 172.21.15.110:0/1320655650 --> v1:172.21.15.110:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f341006b0f0 con 0x7f341006b860 2024-09-15T03:40:49.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.195+0000 7f340dffb700 1 -- 172.21.15.110:0/1320655650 <== mon.0 v1:172.21.15.110:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 644009436 0 0) 0x7f34000017f0 con 0x7f341006b860 2024-09-15T03:40:49.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.195+0000 7f340dffb700 1 -- 172.21.15.110:0/1320655650 >> v1:172.21.15.110:6789/0 conn(0x7f341006b860 legacy=0x7f341006de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:49.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.195+0000 7f340dffb700 1 --2- 172.21.15.110:0/1320655650 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f33f8003a90 0x7f33f8003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:49.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.195+0000 7f340dffb700 1 --2- 172.21.15.110:0/1320655650 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33f8004810 0x7f33f8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:49.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.195+0000 7f340dffb700 1 --2- 172.21.15.110:0/1320655650 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f34100b3030 0x7f33f80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:49.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.195+0000 7f340dffb700 1 -- 172.21.15.110:0/1320655650 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34100ad890 con 0x7f33f8004810 2024-09-15T03:40:49.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.195+0000 7f340dffb700 1 -- 172.21.15.110:0/1320655650 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f800bb10 con 0x7f33f8003a90 2024-09-15T03:40:49.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.195+0000 7f340dffb700 1 -- 172.21.15.110:0/1320655650 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33f800bcd0 con 0x7f34100b3030 2024-09-15T03:40:49.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.195+0000 7f340e7fc700 1 --2- 172.21.15.110:0/1320655650 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33f8004810 0x7f33f8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:49.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.195+0000 7f340f7fe700 1 --2- 172.21.15.110:0/1320655650 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f34100b3030 0x7f33f80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:49.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.195+0000 7f340effd700 1 --2- 172.21.15.110:0/1320655650 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f33f8003a90 0x7f33f8003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:49.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.195+0000 7f340e7fc700 1 -- 172.21.15.110:0/1320655650 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f34100b3030 msgr2=0x7f33f80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:40:49.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.195+0000 7f340e7fc700 1 --2- 172.21.15.110:0/1320655650 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f34100b3030 0x7f33f80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:49.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.195+0000 7f340e7fc700 1 -- 172.21.15.110:0/1320655650 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f33f8003a90 msgr2=0x7f33f8003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:40:49.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.195+0000 7f340e7fc700 1 --2- 172.21.15.110:0/1320655650 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f33f8003a90 0x7f33f8003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:49.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.195+0000 7f340e7fc700 1 -- 172.21.15.110:0/1320655650 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33f8003570 con 0x7f33f8004810 2024-09-15T03:40:49.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.195+0000 7f340f7fe700 1 --2- 172.21.15.110:0/1320655650 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f34100b3030 0x7f33f80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T03:40:49.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.195+0000 7f340e7fc700 1 --2- 172.21.15.110:0/1320655650 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33f8004810 0x7f33f8004c80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3400005bf0 tx=0x7f3400005230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce73dea0f86f287 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:49.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.195+0000 7f340dffb700 1 -- 172.21.15.110:0/1320655650 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3400005560 con 0x7f33f8004810 2024-09-15T03:40:49.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.195+0000 7f340dffb700 1 -- 172.21.15.110:0/1320655650 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f340000d900 con 0x7f33f8004810 2024-09-15T03:40:49.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.195+0000 7f340dffb700 1 -- 172.21.15.110:0/1320655650 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340000dca0 con 0x7f33f8004810 2024-09-15T03:40:49.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.195+0000 7f340ffff700 1 -- 172.21.15.110:0/1320655650 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33f8004810 msgr2=0x7f33f8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:49.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.195+0000 7f340ffff700 1 --2- 172.21.15.110:0/1320655650 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33f8004810 0x7f33f8004c80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3400005bf0 tx=0x7f3400005230 comp rx=0 tx=0).stop 2024-09-15T03:40:49.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.195+0000 7f340f7fe700 1 -- 172.21.15.110:0/1320655650 reap_dead start 2024-09-15T03:40:49.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.195+0000 7f340ffff700 1 -- 172.21.15.110:0/1320655650 shutdown_connections 2024-09-15T03:40:49.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.195+0000 7f340ffff700 1 -- 172.21.15.110:0/1320655650 >> 172.21.15.110:0/1320655650 conn(0x7f34100680f0 msgr2=0x7f34100b26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:40:49.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.195+0000 7f340ffff700 1 -- 172.21.15.110:0/1320655650 shutdown_connections 2024-09-15T03:40:49.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.195+0000 7f340ffff700 1 -- 172.21.15.110:0/1320655650 wait complete. 2024-09-15T03:40:49.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.199+0000 7f340ffff700 1 Processor -- start 2024-09-15T03:40:49.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.199+0000 7f340ffff700 1 -- start start 2024-09-15T03:40:49.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.199+0000 7f340ffff700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f33f8004810 0x7f34100aab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:49.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.199+0000 7f340ffff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f341013df80 0x7f34101398c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:49.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.199+0000 7f340ffff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3410139e00 0x7f34100a88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:49.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.199+0000 7f340ffff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34100aa2f0 con 0x7f3410139e00 2024-09-15T03:40:49.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.199+0000 7f340ffff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34100a9e60 con 0x7f341013df80 2024-09-15T03:40:49.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.199+0000 7f340ffff700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34100ac1e0 con 0x7f33f8004810 2024-09-15T03:40:49.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.199+0000 7f340f7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3410139e00 0x7f34100a88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:49.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.199+0000 7f340effd700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f33f8004810 0x7f34100aab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:49.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.199+0000 7f340f7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3410139e00 0x7f34100a88e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51038/0 (socket says 172.21.15.110:51038) 2024-09-15T03:40:49.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.199+0000 7f340f7fe700 1 -- 172.21.15.110:0/638443834 learned_addr learned my addr 172.21.15.110:0/638443834 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:40:49.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.199+0000 7f340e7fc700 1 --2- 172.21.15.110:0/638443834 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f341013df80 0x7f34101398c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:49.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.199+0000 7f340effd700 1 -- 172.21.15.110:0/638443834 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f341013df80 msgr2=0x7f34101398c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:49.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.199+0000 7f340effd700 1 --2- 172.21.15.110:0/638443834 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f341013df80 0x7f34101398c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:49.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.199+0000 7f340effd700 1 -- 172.21.15.110:0/638443834 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3410139e00 msgr2=0x7f34100a88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:49.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.199+0000 7f340effd700 1 --2- 172.21.15.110:0/638443834 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3410139e00 0x7f34100a88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:49.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.199+0000 7f340effd700 1 -- 172.21.15.110:0/638443834 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f341013e800 con 0x7f33f8004810 2024-09-15T03:40:49.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.199+0000 7f340e7fc700 1 --2- 172.21.15.110:0/638443834 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f341013df80 0x7f34101398c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T03:40:49.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.199+0000 7f340f7fe700 1 --2- 172.21.15.110:0/638443834 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3410139e00 0x7f34100a88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T03:40:49.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.199+0000 7f340effd700 1 --2- 172.21.15.110:0/638443834 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f33f8004810 0x7f34100aab20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f34040119c0 tx=0x7f3404008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:49.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.199+0000 7f33f6ffd700 1 -- 172.21.15.110:0/638443834 <== mon.2 v2:172.21.15.110:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340400f6d0 con 0x7f33f8004810 2024-09-15T03:40:49.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.199+0000 7f33f6ffd700 1 -- 172.21.15.110:0/638443834 <== mon.2 v2:172.21.15.110:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f340400f830 con 0x7f33f8004810 2024-09-15T03:40:49.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.199+0000 7f340ffff700 1 -- 172.21.15.110:0/638443834 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34100a9a10 con 0x7f33f8004810 2024-09-15T03:40:49.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.199+0000 7f33f6ffd700 1 -- 172.21.15.110:0/638443834 <== mon.2 v2:172.21.15.110:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340400fbd0 con 0x7f33f8004810 2024-09-15T03:40:49.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.199+0000 7f340ffff700 1 -- 172.21.15.110:0/638443834 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f341013a3e0 con 0x7f33f8004810 2024-09-15T03:40:49.208 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.203+0000 7f340ffff700 1 -- 172.21.15.110:0/638443834 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34100a9c00 con 0x7f33f8004810 2024-09-15T03:40:49.208 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.203+0000 7f33f6ffd700 1 -- 172.21.15.110:0/638443834 <== mon.2 v2:172.21.15.110:3301/0 4 ==== mgrmap(e 5) v1 ==== 85847+0+0 (secure 0 0 0) 0x7f3404015460 con 0x7f33f8004810 2024-09-15T03:40:49.209 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.203+0000 7f33f6ffd700 1 --2- 172.21.15.110:0/638443834 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f33fc067a80 0x7f33fc069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:49.209 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.203+0000 7f33f6ffd700 1 -- 172.21.15.110:0/638443834 <== mon.2 v2:172.21.15.110:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5641+0+0 (secure 0 0 0) 0x7f3404085990 con 0x7f33f8004810 2024-09-15T03:40:49.209 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.203+0000 7f340e7fc700 1 --2- 172.21.15.110:0/638443834 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f33fc067a80 0x7f33fc069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:49.209 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.203+0000 7f340e7fc700 1 --2- 172.21.15.110:0/638443834 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f33fc067a80 0x7f33fc069f30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3400005270 tx=0x7f3400016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:49.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.211+0000 7f33f6ffd700 1 -- 172.21.15.110:0/638443834 <== mon.2 v2:172.21.15.110:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f3404052ad0 con 0x7f33f8004810 2024-09-15T03:40:49.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.371+0000 7f340ffff700 1 -- 172.21.15.110:0/638443834 --> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f34100721b0 con 0x7f33fc067a80 2024-09-15T03:40:49.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.383+0000 7f33f6ffd700 1 -- 172.21.15.110:0/638443834 <== mgr.4104 v2:172.21.15.110:6832/22190 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+153374 (secure 0 0 0) 0x7f34100721b0 con 0x7f33fc067a80 2024-09-15T03:40:49.389 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-15T03:40:49.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.387+0000 7f340ffff700 1 -- 172.21.15.110:0/638443834 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f33fc067a80 msgr2=0x7f33fc069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:49.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.387+0000 7f340ffff700 1 --2- 172.21.15.110:0/638443834 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f33fc067a80 0x7f33fc069f30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3400005270 tx=0x7f3400016000 comp rx=0 tx=0).stop 2024-09-15T03:40:49.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.387+0000 7f340ffff700 1 -- 172.21.15.110:0/638443834 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f33f8004810 msgr2=0x7f34100aab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:49.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.387+0000 7f340ffff700 1 --2- 172.21.15.110:0/638443834 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f33f8004810 0x7f34100aab20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f34040119c0 tx=0x7f3404008040 comp rx=0 tx=0).stop 2024-09-15T03:40:49.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.387+0000 7f340ffff700 1 -- 172.21.15.110:0/638443834 shutdown_connections 2024-09-15T03:40:49.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.387+0000 7f340ffff700 1 --2- 172.21.15.110:0/638443834 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f33fc067a80 0x7f33fc069f30 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:49.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.387+0000 7f340ffff700 1 --2- 172.21.15.110:0/638443834 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3410139e00 0x7f34100a88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:49.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.387+0000 7f340ffff700 1 --2- 172.21.15.110:0/638443834 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f341013df80 0x7f34101398c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:49.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.387+0000 7f340ffff700 1 --2- 172.21.15.110:0/638443834 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f33f8004810 0x7f34100aab20 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:49.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.387+0000 7f340ffff700 1 -- 172.21.15.110:0/638443834 >> 172.21.15.110:0/638443834 conn(0x7f34100680f0 msgr2=0x7f34100b1200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:40:49.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.387+0000 7f340ffff700 1 -- 172.21.15.110:0/638443834 shutdown_connections 2024-09-15T03:40:49.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.387+0000 7f340ffff700 1 -- 172.21.15.110:0/638443834 wait complete. 2024-09-15T03:40:49.392 INFO:teuthology.orchestra.run.smithi110.stderr:dumped all 2024-09-15T03:40:49.403 INFO:teuthology.orchestra.run.smithi110.stdout:{"pg_ready":true,"pg_map":{"version":143,"stamp":"2024-09-15T03:40:48.676511+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3570763067,"num_objects":883,"num_object_clones":0,"num_object_copies":1766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":883,"num_whiteouts":0,"num_read":2563,"num_read_kb":2563,"num_write":159105,"num_write_kb":5635359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160810,"ondisk_log_size":160810,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":9118160,"kb_used_data":6685256,"kb_used_omap":13,"kb_used_meta":2432818,"kb_avail":745856560,"statfs":{"total":773094113280,"available":763757117440,"internally_reserved":0,"allocated":6845702144,"data_stored":6840192686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2491206336},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003442"},"pg_stats":[{"pgid":"3.1f","version":"22'3025","reported_seq":3035,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T03:37:55.907201+0000","last_change":"2024-09-15T03:36:48.151935+0000","last_active":"2024-09-15T03:37:55.907201+0000","last_peered":"2024-09-15T03:37:55.907201+0000","last_clean":"2024-09-15T03:37:55.907201+0000","last_became_active":"2024-09-15T03:36:48.151625+0000","last_became_peered":"2024-09-15T03:36:48.151625+0000","last_unstale":"2024-09-15T03:37:55.907201+0000","last_undegraded":"2024-09-15T03:37:55.907201+0000","last_fullsized":"2024-09-15T03:37:55.907201+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":3025,"log_dups_size":0,"ondisk_log_size":3025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:02:31.103276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663296,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3019,"num_write_kb":136067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"23'6207","reported_seq":6522,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:40.006171+0000","last_change":"2024-09-15T03:36:48.150055+0000","last_active":"2024-09-15T03:39:40.006171+0000","last_peered":"2024-09-15T03:39:40.006171+0000","last_clean":"2024-09-15T03:39:40.006171+0000","last_became_active":"2024-09-15T03:36:48.149731+0000","last_became_peered":"2024-09-15T03:36:48.149731+0000","last_unstale":"2024-09-15T03:39:40.006171+0000","last_undegraded":"2024-09-15T03:39:40.006171+0000","last_fullsized":"2024-09-15T03:39:40.006171+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":6207,"log_dups_size":0,"ondisk_log_size":6207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:50:59.926991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113377280,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":1022,"num_read_kb":916,"num_write":6394,"num_write_kb":178051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"23'6287","reported_seq":6400,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:46.098160+0000","last_change":"2024-09-15T03:36:48.162431+0000","last_active":"2024-09-15T03:39:46.098160+0000","last_peered":"2024-09-15T03:39:46.098160+0000","last_clean":"2024-09-15T03:39:46.098160+0000","last_became_active":"2024-09-15T03:36:48.162085+0000","last_became_peered":"2024-09-15T03:36:48.162085+0000","last_unstale":"2024-09-15T03:39:46.098160+0000","last_undegraded":"2024-09-15T03:39:46.098160+0000","last_fullsized":"2024-09-15T03:39:46.098160+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":6287,"log_dups_size":0,"ondisk_log_size":6287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:51:54.108247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121898016,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":109,"num_read_kb":107,"num_write":6275,"num_write_kb":177699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"22'4656","reported_seq":4674,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T03:37:56.481330+0000","last_change":"2024-09-15T03:36:48.159640+0000","last_active":"2024-09-15T03:37:56.481330+0000","last_peered":"2024-09-15T03:37:56.481330+0000","last_clean":"2024-09-15T03:37:56.481330+0000","last_became_active":"2024-09-15T03:36:48.159272+0000","last_became_peered":"2024-09-15T03:36:48.159272+0000","last_unstale":"2024-09-15T03:37:56.481330+0000","last_undegraded":"2024-09-15T03:37:56.481330+0000","last_fullsized":"2024-09-15T03:37:56.481330+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":4656,"log_dups_size":0,"ondisk_log_size":4656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:17:25.869833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88866816,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":4665,"num_write_kb":169740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"23'4796","reported_seq":4806,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:32.771695+0000","last_change":"2024-09-15T03:36:48.150332+0000","last_active":"2024-09-15T03:39:32.771695+0000","last_peered":"2024-09-15T03:39:32.771695+0000","last_clean":"2024-09-15T03:39:32.771695+0000","last_became_active":"2024-09-15T03:36:48.150057+0000","last_became_peered":"2024-09-15T03:36:48.150057+0000","last_unstale":"2024-09-15T03:39:32.771695+0000","last_undegraded":"2024-09-15T03:39:32.771695+0000","last_fullsized":"2024-09-15T03:39:32.771695+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":4796,"log_dups_size":0,"ondisk_log_size":4796,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:15:39.747133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117440512,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4750,"num_write_kb":185221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"23'4713","reported_seq":4728,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:38.053359+0000","last_change":"2024-09-15T03:36:48.144892+0000","last_active":"2024-09-15T03:39:38.053359+0000","last_peered":"2024-09-15T03:39:38.053359+0000","last_clean":"2024-09-15T03:39:38.053359+0000","last_became_active":"2024-09-15T03:36:48.144097+0000","last_became_peered":"2024-09-15T03:36:48.144097+0000","last_unstale":"2024-09-15T03:39:38.053359+0000","last_undegraded":"2024-09-15T03:39:38.053359+0000","last_fullsized":"2024-09-15T03:39:38.053359+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":4713,"log_dups_size":0,"ondisk_log_size":4713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:16:01.325609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68681728,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":22,"num_read_kb":16,"num_write":4699,"num_write_kb":143618,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"23'4491","reported_seq":4502,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:50.201809+0000","last_change":"2024-09-15T03:36:48.142399+0000","last_active":"2024-09-15T03:39:50.201809+0000","last_peered":"2024-09-15T03:39:50.201809+0000","last_clean":"2024-09-15T03:39:50.201809+0000","last_became_active":"2024-09-15T03:36:48.142085+0000","last_became_peered":"2024-09-15T03:36:48.142085+0000","last_unstale":"2024-09-15T03:39:50.201809+0000","last_undegraded":"2024-09-15T03:39:50.201809+0000","last_fullsized":"2024-09-15T03:39:50.201809+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":4491,"log_dups_size":0,"ondisk_log_size":4491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T13:00:10.524261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":111872344,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":4448,"num_write_kb":183583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"23'7205","reported_seq":7219,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:50.436933+0000","last_change":"2024-09-15T03:36:48.138540+0000","last_active":"2024-09-15T03:39:50.436933+0000","last_peered":"2024-09-15T03:39:50.436933+0000","last_clean":"2024-09-15T03:39:50.436933+0000","last_became_active":"2024-09-15T03:36:48.138049+0000","last_became_peered":"2024-09-15T03:36:48.138049+0000","last_unstale":"2024-09-15T03:39:50.436933+0000","last_undegraded":"2024-09-15T03:39:50.436933+0000","last_fullsized":"2024-09-15T03:39:50.436933+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":7205,"log_dups_size":0,"ondisk_log_size":7205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T12:37:15.684685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113246720,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":15,"num_read_kb":11,"num_write":7218,"num_write_kb":193736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"23'7862","reported_seq":7879,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:54.842187+0000","last_change":"2024-09-15T03:36:48.150100+0000","last_active":"2024-09-15T03:39:54.842187+0000","last_peered":"2024-09-15T03:39:54.842187+0000","last_clean":"2024-09-15T03:39:54.842187+0000","last_became_active":"2024-09-15T03:36:48.149792+0000","last_became_peered":"2024-09-15T03:36:48.149792+0000","last_unstale":"2024-09-15T03:39:54.842187+0000","last_undegraded":"2024-09-15T03:39:54.842187+0000","last_fullsized":"2024-09-15T03:39:54.842187+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":7862,"log_dups_size":0,"ondisk_log_size":7862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:42:07.633298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":146800640,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":7874,"num_write_kb":194640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"23'2408","reported_seq":2421,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:32.679416+0000","last_change":"2024-09-15T03:36:48.155132+0000","last_active":"2024-09-15T03:39:32.679416+0000","last_peered":"2024-09-15T03:39:32.679416+0000","last_clean":"2024-09-15T03:39:32.679416+0000","last_became_active":"2024-09-15T03:36:48.154800+0000","last_became_peered":"2024-09-15T03:36:48.154800+0000","last_unstale":"2024-09-15T03:39:32.679416+0000","last_undegraded":"2024-09-15T03:39:32.679416+0000","last_fullsized":"2024-09-15T03:39:32.679416+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":2408,"log_dups_size":0,"ondisk_log_size":2408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:54:21.705564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":2388,"num_write_kb":127362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"23'6301","reported_seq":6317,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:48.732682+0000","last_change":"2024-09-15T03:36:48.152499+0000","last_active":"2024-09-15T03:39:48.732682+0000","last_peered":"2024-09-15T03:39:48.732682+0000","last_clean":"2024-09-15T03:39:48.732682+0000","last_became_active":"2024-09-15T03:36:48.152191+0000","last_became_peered":"2024-09-15T03:36:48.152191+0000","last_unstale":"2024-09-15T03:39:48.732682+0000","last_undegraded":"2024-09-15T03:39:48.732682+0000","last_fullsized":"2024-09-15T03:39:48.732682+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":6301,"log_dups_size":0,"ondisk_log_size":6301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T12:12:56.810421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":124518928,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":19,"num_read_kb":14,"num_write":6316,"num_write_kb":168780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"23'5385","reported_seq":5398,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:42.445288+0000","last_change":"2024-09-15T03:36:48.154566+0000","last_active":"2024-09-15T03:39:42.445288+0000","last_peered":"2024-09-15T03:39:42.445288+0000","last_clean":"2024-09-15T03:39:42.445288+0000","last_became_active":"2024-09-15T03:36:48.153926+0000","last_became_peered":"2024-09-15T03:36:48.153926+0000","last_unstale":"2024-09-15T03:39:42.445288+0000","last_undegraded":"2024-09-15T03:39:42.445288+0000","last_fullsized":"2024-09-15T03:39:42.445288+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":5385,"log_dups_size":0,"ondisk_log_size":5385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:19:06.740804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109121016,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":5342,"num_write_kb":144586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:32.448890+0000","last_change":"2024-09-15T03:36:42.814645+0000","last_active":"2024-09-15T03:39:32.448890+0000","last_peered":"2024-09-15T03:39:32.448890+0000","last_clean":"2024-09-15T03:39:32.448890+0000","last_became_active":"2024-09-15T03:36:40.672828+0000","last_became_peered":"2024-09-15T03:36:40.672828+0000","last_unstale":"2024-09-15T03:39:32.448890+0000","last_undegraded":"2024-09-15T03:39:32.448890+0000","last_fullsized":"2024-09-15T03:39:32.448890+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_clean_scrub_stamp":"2024-09-15T03:36:39.639431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T13:40:53.039569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064069099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'5981","reported_seq":6014,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:54.327858+0000","last_change":"2024-09-15T03:36:48.143463+0000","last_active":"2024-09-15T03:39:54.327858+0000","last_peered":"2024-09-15T03:39:54.327858+0000","last_clean":"2024-09-15T03:39:54.327858+0000","last_became_active":"2024-09-15T03:36:48.143177+0000","last_became_peered":"2024-09-15T03:36:48.143177+0000","last_unstale":"2024-09-15T03:39:54.327858+0000","last_undegraded":"2024-09-15T03:39:54.327858+0000","last_fullsized":"2024-09-15T03:39:54.327858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":5981,"log_dups_size":0,"ondisk_log_size":5981,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:20:24.926833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":134639616,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":92,"num_read_kb":80,"num_write":5989,"num_write_kb":192351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T03:36:51.577473+0000","last_change":"2024-09-15T03:36:42.687267+0000","last_active":"2024-09-15T03:36:51.577473+0000","last_peered":"2024-09-15T03:36:51.577473+0000","last_clean":"2024-09-15T03:36:51.577473+0000","last_became_active":"2024-09-15T03:36:40.671677+0000","last_became_peered":"2024-09-15T03:36:40.671677+0000","last_unstale":"2024-09-15T03:36:51.577473+0000","last_undegraded":"2024-09-15T03:36:51.577473+0000","last_fullsized":"2024-09-15T03:36:51.577473+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_clean_scrub_stamp":"2024-09-15T03:36:39.639431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:55:15.493964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000861089,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"23'5400","reported_seq":5410,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:32.770837+0000","last_change":"2024-09-15T03:36:48.152124+0000","last_active":"2024-09-15T03:39:32.770837+0000","last_peered":"2024-09-15T03:39:32.770837+0000","last_clean":"2024-09-15T03:39:32.770837+0000","last_became_active":"2024-09-15T03:36:48.151844+0000","last_became_peered":"2024-09-15T03:36:48.151844+0000","last_unstale":"2024-09-15T03:39:32.770837+0000","last_undegraded":"2024-09-15T03:39:32.770837+0000","last_fullsized":"2024-09-15T03:39:32.770837+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":5400,"log_dups_size":0,"ondisk_log_size":5400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:17:16.444530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":115998720,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5352,"num_write_kb":174984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T03:36:51.579975+0000","last_change":"2024-09-15T03:36:42.854270+0000","last_active":"2024-09-15T03:36:51.579975+0000","last_peered":"2024-09-15T03:36:51.579975+0000","last_clean":"2024-09-15T03:36:51.579975+0000","last_became_active":"2024-09-15T03:36:41.079216+0000","last_became_peered":"2024-09-15T03:36:41.079216+0000","last_unstale":"2024-09-15T03:36:51.579975+0000","last_undegraded":"2024-09-15T03:36:51.579975+0000","last_fullsized":"2024-09-15T03:36:51.579975+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_clean_scrub_stamp":"2024-09-15T03:36:39.639431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:19:37.610607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062568200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"23'7270","reported_seq":7291,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:53.195871+0000","last_change":"2024-09-15T03:36:48.216882+0000","last_active":"2024-09-15T03:39:53.195871+0000","last_peered":"2024-09-15T03:39:53.195871+0000","last_clean":"2024-09-15T03:39:53.195871+0000","last_became_active":"2024-09-15T03:36:48.216337+0000","last_became_peered":"2024-09-15T03:36:48.216337+0000","last_unstale":"2024-09-15T03:39:53.195871+0000","last_undegraded":"2024-09-15T03:39:53.195871+0000","last_fullsized":"2024-09-15T03:39:53.195871+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":7270,"log_dups_size":0,"ondisk_log_size":7270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:27:25.000492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":146931712,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":45,"num_read_kb":34,"num_write":7284,"num_write_kb":246402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T03:36:51.577450+0000","last_change":"2024-09-15T03:36:42.687270+0000","last_active":"2024-09-15T03:36:51.577450+0000","last_peered":"2024-09-15T03:36:51.577450+0000","last_clean":"2024-09-15T03:36:51.577450+0000","last_became_active":"2024-09-15T03:36:41.077266+0000","last_became_peered":"2024-09-15T03:36:41.077266+0000","last_unstale":"2024-09-15T03:36:51.577450+0000","last_undegraded":"2024-09-15T03:36:51.577450+0000","last_fullsized":"2024-09-15T03:36:51.577450+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_clean_scrub_stamp":"2024-09-15T03:36:39.639431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:54:58.435053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067514400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"22'3282","reported_seq":3292,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T03:37:53.949949+0000","last_change":"2024-09-15T03:36:48.151841+0000","last_active":"2024-09-15T03:37:53.949949+0000","last_peered":"2024-09-15T03:37:53.949949+0000","last_clean":"2024-09-15T03:37:53.949949+0000","last_became_active":"2024-09-15T03:36:48.151520+0000","last_became_peered":"2024-09-15T03:36:48.151520+0000","last_unstale":"2024-09-15T03:37:53.949949+0000","last_undegraded":"2024-09-15T03:37:53.949949+0000","last_fullsized":"2024-09-15T03:37:53.949949+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":3282,"log_dups_size":0,"ondisk_log_size":3282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:24:47.104486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3278,"num_write_kb":144390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"22'3449","reported_seq":3473,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T03:37:58.123758+0000","last_change":"2024-09-15T03:36:48.146454+0000","last_active":"2024-09-15T03:37:58.123758+0000","last_peered":"2024-09-15T03:37:58.123758+0000","last_clean":"2024-09-15T03:37:58.123758+0000","last_became_active":"2024-09-15T03:36:48.145845+0000","last_became_peered":"2024-09-15T03:36:48.145845+0000","last_unstale":"2024-09-15T03:37:58.123758+0000","last_undegraded":"2024-09-15T03:37:58.123758+0000","last_fullsized":"2024-09-15T03:37:58.123758+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":3449,"log_dups_size":0,"ondisk_log_size":3449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:46:57.211180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113246208,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":50,"num_read_kb":41,"num_write":3479,"num_write_kb":144783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T03:36:50.383528+0000","last_change":"2024-09-15T03:36:42.695885+0000","last_active":"2024-09-15T03:36:50.383528+0000","last_peered":"2024-09-15T03:36:50.383528+0000","last_clean":"2024-09-15T03:36:50.383528+0000","last_became_active":"2024-09-15T03:36:40.673743+0000","last_became_peered":"2024-09-15T03:36:40.673743+0000","last_unstale":"2024-09-15T03:36:50.383528+0000","last_undegraded":"2024-09-15T03:36:50.383528+0000","last_fullsized":"2024-09-15T03:36:50.383528+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_clean_scrub_stamp":"2024-09-15T03:36:39.639431+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T13:57:31.462981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002159518,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"23'4710","reported_seq":4734,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:32.771799+0000","last_change":"2024-09-15T03:36:48.216887+0000","last_active":"2024-09-15T03:39:32.771799+0000","last_peered":"2024-09-15T03:39:32.771799+0000","last_clean":"2024-09-15T03:39:32.771799+0000","last_became_active":"2024-09-15T03:36:48.216269+0000","last_became_peered":"2024-09-15T03:36:48.216269+0000","last_unstale":"2024-09-15T03:39:32.771799+0000","last_undegraded":"2024-09-15T03:39:32.771799+0000","last_fullsized":"2024-09-15T03:39:32.771799+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":4710,"log_dups_size":0,"ondisk_log_size":4710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T12:10:11.804094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":142877200,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":51,"num_read_kb":164,"num_write":4698,"num_write_kb":182422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:32.402704+0000","last_change":"2024-09-15T03:36:42.801569+0000","last_active":"2024-09-15T03:39:32.402704+0000","last_peered":"2024-09-15T03:39:32.402704+0000","last_clean":"2024-09-15T03:39:32.402704+0000","last_became_active":"2024-09-15T03:36:40.669392+0000","last_became_peered":"2024-09-15T03:36:40.669392+0000","last_unstale":"2024-09-15T03:39:32.402704+0000","last_undegraded":"2024-09-15T03:39:32.402704+0000","last_fullsized":"2024-09-15T03:39:32.402704+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_clean_scrub_stamp":"2024-09-15T03:36:39.639431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:00:33.102146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059629799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"23'3227","reported_seq":3245,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:32.683131+0000","last_change":"2024-09-15T03:36:48.144585+0000","last_active":"2024-09-15T03:39:32.683131+0000","last_peered":"2024-09-15T03:39:32.683131+0000","last_clean":"2024-09-15T03:39:32.683131+0000","last_became_active":"2024-09-15T03:36:48.144313+0000","last_became_peered":"2024-09-15T03:36:48.144313+0000","last_unstale":"2024-09-15T03:39:32.683131+0000","last_undegraded":"2024-09-15T03:39:32.683131+0000","last_fullsized":"2024-09-15T03:39:32.683131+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":3227,"log_dups_size":0,"ondisk_log_size":3227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T15:27:39.715172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92405760,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":33,"num_read_kb":87,"num_write":3327,"num_write_kb":128260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T03:36:51.579392+0000","last_change":"2024-09-15T03:36:42.854222+0000","last_active":"2024-09-15T03:36:51.579392+0000","last_peered":"2024-09-15T03:36:51.579392+0000","last_clean":"2024-09-15T03:36:51.579392+0000","last_became_active":"2024-09-15T03:36:40.672416+0000","last_became_peered":"2024-09-15T03:36:40.672416+0000","last_unstale":"2024-09-15T03:36:51.579392+0000","last_undegraded":"2024-09-15T03:36:51.579392+0000","last_fullsized":"2024-09-15T03:36:51.579392+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_clean_scrub_stamp":"2024-09-15T03:36:39.639431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T15:08:21.420921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065187600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"23'3107","reported_seq":3117,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:46.717682+0000","last_change":"2024-09-15T03:36:48.151148+0000","last_active":"2024-09-15T03:39:46.717682+0000","last_peered":"2024-09-15T03:39:46.717682+0000","last_clean":"2024-09-15T03:39:46.717682+0000","last_became_active":"2024-09-15T03:36:48.150693+0000","last_became_peered":"2024-09-15T03:36:48.150693+0000","last_unstale":"2024-09-15T03:39:46.717682+0000","last_undegraded":"2024-09-15T03:39:46.717682+0000","last_fullsized":"2024-09-15T03:39:46.717682+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":3107,"log_dups_size":0,"ondisk_log_size":3107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:27:52.876606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67108864,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3105,"num_write_kb":103402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T03:36:50.383548+0000","last_change":"2024-09-15T03:36:42.694486+0000","last_active":"2024-09-15T03:36:50.383548+0000","last_peered":"2024-09-15T03:36:50.383548+0000","last_clean":"2024-09-15T03:36:50.383548+0000","last_became_active":"2024-09-15T03:36:40.674442+0000","last_became_peered":"2024-09-15T03:36:40.674442+0000","last_unstale":"2024-09-15T03:36:50.383548+0000","last_undegraded":"2024-09-15T03:36:50.383548+0000","last_fullsized":"2024-09-15T03:36:50.383548+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_clean_scrub_stamp":"2024-09-15T03:36:39.639431+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:25:57.474430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067001500000000004,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":106,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T03:36:51.579500+0000","last_change":"2024-09-15T03:36:38.815652+0000","last_active":"2024-09-15T03:36:51.579500+0000","last_peered":"2024-09-15T03:36:51.579500+0000","last_clean":"2024-09-15T03:36:51.579500+0000","last_became_active":"2024-09-15T03:36:38.813490+0000","last_became_peered":"2024-09-15T03:36:38.813490+0000","last_unstale":"2024-09-15T03:36:51.579500+0000","last_undegraded":"2024-09-15T03:36:51.579500+0000","last_fullsized":"2024-09-15T03:36:51.579500+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:37.618040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:37.618040+0000","last_clean_scrub_stamp":"2024-09-15T03:36:37.618040+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:02:12.095255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"23'5187","reported_seq":5201,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:32.773823+0000","last_change":"2024-09-15T03:36:48.142652+0000","last_active":"2024-09-15T03:39:32.773823+0000","last_peered":"2024-09-15T03:39:32.773823+0000","last_clean":"2024-09-15T03:39:32.773823+0000","last_became_active":"2024-09-15T03:36:48.142313+0000","last_became_peered":"2024-09-15T03:36:48.142313+0000","last_unstale":"2024-09-15T03:39:32.773823+0000","last_undegraded":"2024-09-15T03:39:32.773823+0000","last_fullsized":"2024-09-15T03:39:32.773823+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":5187,"log_dups_size":0,"ondisk_log_size":5187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T13:50:10.152068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":116760576,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":12,"num_read_kb":135,"num_write":5130,"num_write_kb":177200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"23'7935","reported_seq":8051,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:49.720540+0000","last_change":"2024-09-15T03:36:48.218591+0000","last_active":"2024-09-15T03:39:49.720540+0000","last_peered":"2024-09-15T03:39:49.720540+0000","last_clean":"2024-09-15T03:39:49.720540+0000","last_became_active":"2024-09-15T03:36:48.218083+0000","last_became_peered":"2024-09-15T03:36:48.218083+0000","last_unstale":"2024-09-15T03:39:49.720540+0000","last_undegraded":"2024-09-15T03:39:49.720540+0000","last_fullsized":"2024-09-15T03:39:49.720540+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":7935,"log_dups_size":0,"ondisk_log_size":7935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:15:32.571275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663808,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":356,"num_read_kb":379,"num_write":5975,"num_write_kb":152245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"24'5837","reported_seq":5868,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:48.324243+0000","last_change":"2024-09-15T03:36:48.155747+0000","last_active":"2024-09-15T03:40:48.324243+0000","last_peered":"2024-09-15T03:40:48.324243+0000","last_clean":"2024-09-15T03:40:48.324243+0000","last_became_active":"2024-09-15T03:36:48.155263+0000","last_became_peered":"2024-09-15T03:36:48.155263+0000","last_unstale":"2024-09-15T03:40:48.324243+0000","last_undegraded":"2024-09-15T03:40:48.324243+0000","last_fullsized":"2024-09-15T03:40:48.324243+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":5837,"log_dups_size":0,"ondisk_log_size":5837,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:40:42.479597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":137103712,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":68,"num_read_kb":57,"num_write":5841,"num_write_kb":205281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"22'3150","reported_seq":3160,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T03:37:55.878205+0000","last_change":"2024-09-15T03:36:48.138000+0000","last_active":"2024-09-15T03:37:55.878205+0000","last_peered":"2024-09-15T03:37:55.878205+0000","last_clean":"2024-09-15T03:37:55.878205+0000","last_became_active":"2024-09-15T03:36:48.137589+0000","last_became_peered":"2024-09-15T03:36:48.137589+0000","last_unstale":"2024-09-15T03:37:55.878205+0000","last_undegraded":"2024-09-15T03:37:55.878205+0000","last_fullsized":"2024-09-15T03:37:55.878205+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":3150,"log_dups_size":0,"ondisk_log_size":3150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:02:31.208417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3149,"num_write_kb":152324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"23'4730","reported_seq":4757,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:32.766647+0000","last_change":"2024-09-15T03:36:48.151666+0000","last_active":"2024-09-15T03:39:32.766647+0000","last_peered":"2024-09-15T03:39:32.766647+0000","last_clean":"2024-09-15T03:39:32.766647+0000","last_became_active":"2024-09-15T03:36:48.151365+0000","last_became_peered":"2024-09-15T03:36:48.151365+0000","last_unstale":"2024-09-15T03:39:32.766647+0000","last_undegraded":"2024-09-15T03:39:32.766647+0000","last_fullsized":"2024-09-15T03:39:32.766647+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":4730,"log_dups_size":0,"ondisk_log_size":4730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T15:05:31.496486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121635328,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":72,"num_read_kb":60,"num_write":4709,"num_write_kb":215880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"23'5517","reported_seq":5531,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:32.678901+0000","last_change":"2024-09-15T03:36:48.155332+0000","last_active":"2024-09-15T03:39:32.678901+0000","last_peered":"2024-09-15T03:39:32.678901+0000","last_clean":"2024-09-15T03:39:32.678901+0000","last_became_active":"2024-09-15T03:36:48.154955+0000","last_became_peered":"2024-09-15T03:36:48.154955+0000","last_unstale":"2024-09-15T03:39:32.678901+0000","last_undegraded":"2024-09-15T03:39:32.678901+0000","last_fullsized":"2024-09-15T03:39:32.678901+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":5517,"log_dups_size":0,"ondisk_log_size":5517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:17:29.191405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117440512,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":5499,"num_write_kb":248958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"23'4086","reported_seq":4166,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:32.831744+0000","last_change":"2024-09-15T03:36:48.162708+0000","last_active":"2024-09-15T03:39:32.831744+0000","last_peered":"2024-09-15T03:39:32.831744+0000","last_clean":"2024-09-15T03:39:32.831744+0000","last_became_active":"2024-09-15T03:36:48.162401+0000","last_became_peered":"2024-09-15T03:36:48.162401+0000","last_unstale":"2024-09-15T03:39:32.831744+0000","last_undegraded":"2024-09-15T03:39:32.831744+0000","last_fullsized":"2024-09-15T03:39:32.831744+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":4086,"log_dups_size":0,"ondisk_log_size":4086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:15:25.156037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104866320,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":269,"num_read_kb":201,"num_write":4343,"num_write_kb":177773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"23'3614","reported_seq":3643,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:32.772767+0000","last_change":"2024-09-15T03:36:48.162180+0000","last_active":"2024-09-15T03:39:32.772767+0000","last_peered":"2024-09-15T03:39:32.772767+0000","last_clean":"2024-09-15T03:39:32.772767+0000","last_became_active":"2024-09-15T03:36:48.161897+0000","last_became_peered":"2024-09-15T03:36:48.161897+0000","last_unstale":"2024-09-15T03:39:32.772767+0000","last_undegraded":"2024-09-15T03:39:32.772767+0000","last_fullsized":"2024-09-15T03:39:32.772767+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":3614,"log_dups_size":0,"ondisk_log_size":3614,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:44:27.564730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":130023424,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":75,"num_read_kb":60,"num_write":3605,"num_write_kb":185220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"24'4055","reported_seq":4066,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:48.332171+0000","last_change":"2024-09-15T03:36:48.149972+0000","last_active":"2024-09-15T03:40:48.332171+0000","last_peered":"2024-09-15T03:40:48.332171+0000","last_clean":"2024-09-15T03:40:48.332171+0000","last_became_active":"2024-09-15T03:36:48.149651+0000","last_became_peered":"2024-09-15T03:36:48.149651+0000","last_unstale":"2024-09-15T03:40:48.332171+0000","last_undegraded":"2024-09-15T03:40:48.332171+0000","last_fullsized":"2024-09-15T03:40:48.332171+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":4055,"log_dups_size":0,"ondisk_log_size":4055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T15:32:38.282915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":111869952,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4006,"num_write_kb":198851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"22'2797","reported_seq":2823,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T03:37:53.771805+0000","last_change":"2024-09-15T03:36:48.159682+0000","last_active":"2024-09-15T03:37:53.771805+0000","last_peered":"2024-09-15T03:37:53.771805+0000","last_clean":"2024-09-15T03:37:53.771805+0000","last_became_active":"2024-09-15T03:36:48.159380+0000","last_became_peered":"2024-09-15T03:36:48.159380+0000","last_unstale":"2024-09-15T03:37:53.771805+0000","last_undegraded":"2024-09-15T03:37:53.771805+0000","last_fullsized":"2024-09-15T03:37:53.771805+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":2797,"log_dups_size":0,"ondisk_log_size":2797,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T13:43:53.575032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":50,"num_read_kb":41,"num_write":2819,"num_write_kb":144138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"23'5712","reported_seq":5727,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:32.775461+0000","last_change":"2024-09-15T03:36:48.151085+0000","last_active":"2024-09-15T03:39:32.775461+0000","last_peered":"2024-09-15T03:39:32.775461+0000","last_clean":"2024-09-15T03:39:32.775461+0000","last_became_active":"2024-09-15T03:36:48.150784+0000","last_became_peered":"2024-09-15T03:36:48.150784+0000","last_unstale":"2024-09-15T03:39:32.775461+0000","last_undegraded":"2024-09-15T03:39:32.775461+0000","last_fullsized":"2024-09-15T03:39:32.775461+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":5712,"log_dups_size":0,"ondisk_log_size":5712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:11:11.620798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":138543104,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":22,"num_read_kb":16,"num_write":5677,"num_write_kb":255624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"23'8353","reported_seq":8372,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:46.094057+0000","last_change":"2024-09-15T03:36:48.159758+0000","last_active":"2024-09-15T03:39:46.094057+0000","last_peered":"2024-09-15T03:39:46.094057+0000","last_clean":"2024-09-15T03:39:46.094057+0000","last_became_active":"2024-09-15T03:36:48.159470+0000","last_became_peered":"2024-09-15T03:36:48.159470+0000","last_unstale":"2024-09-15T03:39:46.094057+0000","last_undegraded":"2024-09-15T03:39:46.094057+0000","last_fullsized":"2024-09-15T03:39:46.094057+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":8353,"log_dups_size":0,"ondisk_log_size":8353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:06:20.271144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":126026248,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":36,"num_read_kb":28,"num_write":8349,"num_write_kb":200158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":3570172680,"num_objects":879,"num_object_clones":0,"num_object_copies":1758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":879,"num_whiteouts":0,"num_read":2497,"num_read_kb":2508,"num_write":159002,"num_write_kb":5633729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":6842703872,"data_stored":6838429848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160735,"ondisk_log_size":160735,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542196,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1357616,"kb_used_data":1047784,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":93014224,"statfs":{"total":96636764160,"available":95246565376,"internally_reserved":0,"allocated":1072930816,"data_stored":1072152433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 03:39:47 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":2.0049999999999999,"15min":2.5830000000000002},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.1799999999999999,"5min":35.189,"15min":35.189},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.749,"15min":0.74199999999999999},"min":{"1min":0.39900000000000002,"5min":0.245,"15min":0.245},"max":{"1min":1.173,"5min":1.427,"15min":1.427},"last":0.76500000000000001}]},{"osd":1,"last update":"Sun Sep 15 03:39:47 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":2.0150000000000001,"15min":2.544},"min":{"1min":0.315,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.54,"5min":35.223999999999997,"15min":35.223999999999997},"last":1.1779999999999999},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78700000000000003,"15min":0.76700000000000002},"min":{"1min":0.38300000000000001,"5min":0.308,"15min":0.308},"max":{"1min":1.1930000000000001,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.84799999999999998}]},{"osd":2,"last update":"Sun Sep 15 03:39:47 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.73199999999999998,"15min":0.72899999999999998},"min":{"1min":0.374,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.2509999999999999,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":1.0409999999999999},{"interface":"front","average":{"1min":0.81799999999999995,"5min":2.105,"15min":2.641},"min":{"1min":0.40899999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.3879999999999999,"5min":35.283000000000001,"15min":35.283000000000001},"last":1.0149999999999999}]},{"osd":3,"last update":"Sun Sep 15 03:39:47 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.78400000000000003,"15min":0.74099999999999999},"min":{"1min":0.45500000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.266,"5min":1.71,"15min":1.71},"last":1.117},{"interface":"front","average":{"1min":0.92000000000000004,"5min":2.1749999999999998,"15min":2.6930000000000001},"min":{"1min":0.50800000000000001,"5min":0.378,"15min":0.378},"max":{"1min":1.3580000000000001,"5min":35.255000000000003,"15min":35.255000000000003},"last":0.97999999999999998}]},{"osd":4,"last update":"Sun Sep 15 03:39:47 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":1.048,"15min":0.80800000000000005},"min":{"1min":0.33900000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.3380000000000001,"5min":25.52,"15min":25.52},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.81699999999999995,"15min":0.77300000000000002},"min":{"1min":0.32200000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.339,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":1.1499999999999999}]},{"osd":5,"last update":"Sun Sep 15 03:39:47 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":2.238,"15min":2.778},"min":{"1min":0.41599999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.329,"5min":35.070999999999998,"15min":35.070999999999998},"last":1.2290000000000001},{"interface":"front","average":{"1min":0.96999999999999997,"5min":1.0820000000000001,"15min":1.101},"min":{"1min":0.43099999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.4430000000000001,"5min":5.2999999999999998,"15min":5.2999999999999998},"last":1.2}]},{"osd":6,"last update":"Sun Sep 15 03:39:47 2024","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.94399999999999995,"15min":0.878},"min":{"1min":0.42099999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.355,"5min":2.5449999999999999,"15min":2.5449999999999999},"last":1.085},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.90700000000000003,"15min":0.84699999999999998},"min":{"1min":0.40799999999999997,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.484,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":1.452}]}]},{"osd":6,"up_from":14,"seq":60129542196,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":651352,"kb_used_data":349836,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93720488,"statfs":{"total":96636764160,"available":95969779712,"internally_reserved":0,"allocated":358232064,"data_stored":357835405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 03:40:43 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":1.0049999999999999,"15min":1.155},"min":{"1min":0.253,"5min":0.25,"15min":0.25},"max":{"1min":1.208,"5min":7.3170000000000002,"15min":7.3170000000000002},"last":1.3069999999999999},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.82399999999999995,"15min":0.77300000000000002},"min":{"1min":0.47799999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.212,"5min":7.3849999999999998,"15min":7.3849999999999998},"last":1.157}]},{"osd":1,"last update":"Sun Sep 15 03:40:43 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.72899999999999998,"15min":0.69099999999999995},"min":{"1min":0.30299999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.157,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.89600000000000002,"15min":0.91800000000000004},"min":{"1min":0.45800000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.304,"5min":7.3559999999999999,"15min":7.3559999999999999},"last":1.1100000000000001}]},{"osd":2,"last update":"Sun Sep 15 03:40:43 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.92800000000000005,"15min":0.93500000000000005},"min":{"1min":0.436,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.286,"5min":7.4960000000000004,"15min":7.4960000000000004},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.84699999999999998,"15min":0.88600000000000001},"min":{"1min":0.49199999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.1499999999999999,"5min":1.575,"15min":1.575},"last":1.474}]},{"osd":3,"last update":"Sun Sep 15 03:40:43 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":1.0569999999999999,"15min":1.1499999999999999},"min":{"1min":0.42399999999999999,"5min":0.308,"15min":0.308},"max":{"1min":1.236,"5min":7.5869999999999997,"15min":7.5869999999999997},"last":0.89600000000000002},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.2869999999999999,"15min":1.526},"min":{"1min":0.53800000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.423,"5min":11.494,"15min":11.494},"last":1.3819999999999999}]},{"osd":4,"last update":"Sun Sep 15 03:40:43 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":1.0449999999999999,"15min":1.0820000000000001},"min":{"1min":0.32600000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.319,"5min":10.348000000000001,"15min":10.348000000000001},"last":1.212},{"interface":"front","average":{"1min":1.083,"5min":1.0509999999999999,"15min":1.0049999999999999},"min":{"1min":0.44,"5min":0.316,"15min":0.316},"max":{"1min":1.373,"5min":7.5620000000000003,"15min":7.5620000000000003},"last":0.92900000000000005}]},{"osd":5,"last update":"Sun Sep 15 03:40:43 2024","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.86899999999999999,"15min":0.83799999999999997},"min":{"1min":0.54500000000000004,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.3400000000000001,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":1.5349999999999999},{"interface":"front","average":{"1min":1.147,"5min":1.0149999999999999,"15min":0.96699999999999997},"min":{"1min":0.56699999999999995,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.444,"5min":1.623,"15min":1.623},"last":1.5780000000000001}]},{"osd":7,"last update":"Sun Sep 15 03:40:43 2024","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.121,"15min":1.0149999999999999},"min":{"1min":0.40500000000000003,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.4279999999999999,"5min":10.923999999999999,"15min":10.923999999999999},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.1160000000000001,"15min":1.01},"min":{"1min":0.52100000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.7130000000000001,"5min":10.74,"15min":10.74},"last":1.0960000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542195,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1179920,"kb_used_data":874248,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":93191920,"statfs":{"total":96636764160,"available":95428526080,"internally_reserved":0,"allocated":895229952,"data_stored":894483388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 03:39:45 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.68300000000000005,"15min":0.65900000000000003},"min":{"1min":0.30399999999999999,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.80800000000000005,"15min":0.83999999999999997},"min":{"1min":0.30199999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.139,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":0.81299999999999994}]},{"osd":1,"last update":"Sun Sep 15 03:39:45 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.746,"15min":0.70399999999999996},"min":{"1min":0.32900000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.1259999999999999,"5min":2.302,"15min":2.302},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.71099999999999997,"15min":0.68899999999999995},"min":{"1min":0.44600000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.23,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.90000000000000002}]},{"osd":2,"last update":"Sun Sep 15 03:39:45 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.82199999999999995,"15min":0.82399999999999995},"min":{"1min":0.54500000000000004,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.0800000000000001,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85399999999999998,"15min":0.85999999999999999},"min":{"1min":0.38300000000000001,"5min":0.314,"15min":0.314},"max":{"1min":1.3839999999999999,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":1.0249999999999999}]},{"osd":3,"last update":"Sun Sep 15 03:39:45 2024","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":1.669,"15min":2.0310000000000001},"min":{"1min":0.46700000000000003,"5min":0.375,"15min":0.375},"max":{"1min":1.2629999999999999,"5min":26.664999999999999,"15min":26.664999999999999},"last":1.073},{"interface":"front","average":{"1min":0.98199999999999998,"5min":1.827,"15min":2.2189999999999999},"min":{"1min":0.49399999999999999,"5min":0.312,"15min":0.312},"max":{"1min":1.5069999999999999,"5min":26.748000000000001,"15min":26.748000000000001},"last":1.111}]},{"osd":4,"last update":"Sun Sep 15 03:39:45 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.90000000000000002,"15min":0.88},"min":{"1min":0.34999999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.4510000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":1.1830000000000001},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.90500000000000003,"15min":0.875},"min":{"1min":0.47799999999999998,"5min":0.318,"15min":0.318},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":1.222}]},{"osd":6,"last update":"Sun Sep 15 03:39:45 2024","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":0.97499999999999998,"15min":0.95999999999999996},"min":{"1min":0.55700000000000005,"5min":0.373,"15min":0.373},"max":{"1min":1.6020000000000001,"5min":1.9730000000000001,"15min":1.9730000000000001},"last":0.97699999999999998},{"interface":"front","average":{"1min":1.121,"5min":1.026,"15min":1},"min":{"1min":0.439,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.5369999999999999,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.2689999999999999}]},{"osd":7,"last update":"Sun Sep 15 03:39:45 2024","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.94299999999999995,"15min":0.90200000000000002},"min":{"1min":0.42199999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.4470000000000001,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":1.29},{"interface":"front","average":{"1min":1.252,"5min":1.1020000000000001,"15min":1.071},"min":{"1min":0.54400000000000004,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":4.3070000000000004,"5min":4.3070000000000004,"15min":4.3070000000000004},"last":1.3400000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542195,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1279268,"kb_used_data":981916,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93092572,"statfs":{"total":96636764160,"available":95326793728,"internally_reserved":0,"allocated":1005481984,"data_stored":1004653769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 03:39:41 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.64700000000000002,"15min":0.69899999999999995},"min":{"1min":0.27400000000000002,"5min":0.22,"15min":0.22},"max":{"1min":1.0509999999999999,"5min":4.7809999999999997,"15min":4.7809999999999997},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.85899999999999999,"15min":0.99099999999999999},"min":{"1min":0.314,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.98899999999999999,"5min":7.7370000000000001,"15min":7.7370000000000001},"last":0.95899999999999996}]},{"osd":1,"last update":"Sun Sep 15 03:39:41 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.746,"15min":0.627},"min":{"1min":0.27600000000000002,"5min":0.217,"15min":0.217},"max":{"1min":1.002,"5min":13.749000000000001,"15min":13.749000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.68000000000000005,"5min":1.29,"15min":0.96599999999999997},"min":{"1min":0.31900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.052,"5min":46.225999999999999,"15min":46.225999999999999},"last":1.1419999999999999}]},{"osd":2,"last update":"Sun Sep 15 03:39:41 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.71699999999999997,"15min":0.69899999999999995},"min":{"1min":0.39100000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.1379999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.72799999999999998,"15min":0.71499999999999997},"min":{"1min":0.45400000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":1.0760000000000001}]},{"osd":4,"last update":"Sun Sep 15 03:39:41 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.92600000000000005,"15min":0.94199999999999995},"min":{"1min":0.40799999999999997,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.335,"5min":2.9870000000000001,"15min":2.9870000000000001},"last":0.877},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.94299999999999995,"15min":0.96499999999999997},"min":{"1min":0.42599999999999999,"5min":0.376,"15min":0.376},"max":{"1min":1.3080000000000001,"5min":2.609,"15min":2.609},"last":1.3109999999999999}]},{"osd":5,"last update":"Sun Sep 15 03:39:41 2024","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.86599999999999999,"15min":0.84099999999999997},"min":{"1min":0.47999999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.22,"5min":1.381,"15min":1.381},"last":1.52},{"interface":"front","average":{"1min":0.86099999999999999,"5min":1.0649999999999999,"15min":1.1040000000000001},"min":{"1min":0.40100000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.242,"5min":5.0309999999999997,"15min":5.0309999999999997},"last":1.284}]},{"osd":6,"last update":"Sun Sep 15 03:39:41 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.97599999999999998,"15min":0.97299999999999998},"min":{"1min":0.54600000000000004,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.3200000000000001,"5min":1.516,"15min":1.516},"last":1.3899999999999999},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.97499999999999998,"15min":0.96299999999999997},"min":{"1min":0.59699999999999998,"5min":0.503,"15min":0.503},"max":{"1min":1.5680000000000001,"5min":2.5550000000000002,"15min":2.5550000000000002},"last":1.115}]},{"osd":7,"last update":"Sun Sep 15 03:39:41 2024","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":1.1359999999999999,"15min":1.1850000000000001},"min":{"1min":0.57799999999999996,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.494,"5min":5.5439999999999996,"15min":5.5439999999999996},"last":1.036},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.399,"15min":1.5600000000000001},"min":{"1min":0.56999999999999995,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.357,"5min":9.5850000000000009,"15min":9.5850000000000009},"last":1.466}]}]},{"osd":2,"up_from":14,"seq":60129542197,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":683092,"kb_used_data":377416,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":93688748,"statfs":{"total":96636764160,"available":95937277952,"internally_reserved":0,"allocated":386473984,"data_stored":386097365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 03:40:46 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.59699999999999998,"15min":0.61199999999999999},"min":{"1min":0.39800000000000002,"5min":0.193,"15min":0.193},"max":{"1min":0.96499999999999997,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.66700000000000004,"15min":0.65700000000000003},"min":{"1min":0.54000000000000004,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":1.228,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.59299999999999997}]},{"osd":1,"last update":"Sun Sep 15 03:40:46 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.73099999999999998,"15min":0.72599999999999998},"min":{"1min":0.47299999999999998,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.044,"5min":1.258,"15min":1.258},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.61599999999999999,"15min":0.57999999999999996},"min":{"1min":0.41999999999999998,"5min":0.222,"15min":0.222},"max":{"1min":0.92800000000000005,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.71599999999999997}]},{"osd":3,"last update":"Sun Sep 15 03:40:46 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.68300000000000005,"15min":0.66000000000000003},"min":{"1min":0.42399999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.98299999999999998,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.77800000000000002,"15min":0.73399999999999999},"min":{"1min":0.40100000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.753}]},{"osd":4,"last update":"Sun Sep 15 03:40:46 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.84099999999999997,"15min":0.77900000000000003},"min":{"1min":0.55500000000000005,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.02,"5min":0.91100000000000003,"15min":0.86599999999999999},"min":{"1min":0.53100000000000003,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.4239999999999999,"5min":1.573,"15min":1.573},"last":0.92000000000000004}]},{"osd":5,"last update":"Sun Sep 15 03:40:46 2024","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.94299999999999995,"15min":0.95399999999999996},"min":{"1min":0.68500000000000005,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.3680000000000001,"5min":4.5839999999999996,"15min":4.5839999999999996},"last":0.95299999999999996},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0549999999999999,"15min":1.113},"min":{"1min":0.50900000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.341,"5min":4.8230000000000004,"15min":4.8230000000000004},"last":1.2430000000000001}]},{"osd":6,"last update":"Sun Sep 15 03:40:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":0.94199999999999995,"15min":0.86799999999999999},"min":{"1min":0.81100000000000005,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.288,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":0.88700000000000001},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.03,"15min":0.96999999999999997},"min":{"1min":0.49299999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.425,"5min":1.9039999999999999,"15min":1.9039999999999999},"last":1.0900000000000001}]},{"osd":7,"last update":"Sun Sep 15 03:40:46 2024","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.0389999999999999,"15min":1.044},"min":{"1min":0.72599999999999998,"5min":0.379,"15min":0.379},"max":{"1min":1.4590000000000001,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":1.331},{"interface":"front","average":{"1min":1.169,"5min":1.3540000000000001,"15min":1.6080000000000001},"min":{"1min":0.74099999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.4770000000000001,"5min":10.428000000000001,"15min":10.428000000000001},"last":1.022}]}]},{"osd":4,"up_from":13,"seq":55834574901,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1618136,"kb_used_data":1320784,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":92753704,"statfs":{"total":96636764160,"available":94979792896,"internally_reserved":0,"allocated":1352482816,"data_stored":1351530757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 03:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.85799999999999998,"15min":0.93999999999999995},"min":{"1min":0.311,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.137,"5min":6.5010000000000003,"15min":6.5010000000000003},"last":1.056},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.85299999999999998,"15min":0.89300000000000002},"min":{"1min":0.34300000000000003,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.159,"5min":3.645,"15min":3.645},"last":0.80100000000000005}]},{"osd":1,"last update":"Sun Sep 15 03:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.84499999999999997,"15min":0.88900000000000001},"min":{"1min":0.29299999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.2589999999999999,"5min":3.7069999999999999,"15min":3.7069999999999999},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.81699999999999995,"15min":0.80200000000000005},"min":{"1min":0.44600000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.272,"5min":2.5779999999999998,"15min":2.5779999999999998},"last":1.093}]},{"osd":2,"last update":"Sun Sep 15 03:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.84999999999999998,"15min":0.81100000000000005},"min":{"1min":0.45000000000000001,"5min":0.311,"15min":0.311},"max":{"1min":1.212,"5min":2.766,"15min":2.766},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.84199999999999997,"15min":0.77200000000000002},"min":{"1min":0.375,"5min":0.316,"15min":0.316},"max":{"1min":1.256,"5min":3.2749999999999999,"15min":3.2749999999999999},"last":0.90600000000000003}]},{"osd":3,"last update":"Sun Sep 15 03:40:45 2024","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.0609999999999999,"15min":1.173},"min":{"1min":0.501,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.357,"5min":6.3620000000000001,"15min":6.3620000000000001},"last":0.97699999999999998},{"interface":"front","average":{"1min":1.0640000000000001,"5min":0.94899999999999995,"15min":0.87},"min":{"1min":0.58799999999999997,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.4330000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":1.1759999999999999}]},{"osd":5,"last update":"Sun Sep 15 03:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":1.0069999999999999,"15min":1.0629999999999999},"min":{"1min":0.41899999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.6890000000000001,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":1.127},{"interface":"front","average":{"1min":1.171,"5min":1.143,"15min":1.107},"min":{"1min":0.53000000000000003,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.651,"5min":4.1710000000000003,"15min":4.1710000000000003},"last":1.3200000000000001}]},{"osd":6,"last update":"Sun Sep 15 03:40:45 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.0309999999999999,"15min":1.0389999999999999},"min":{"1min":0.45800000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.6120000000000001,"5min":3.669,"15min":3.669},"last":1.1990000000000001},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.0609999999999999,"15min":0.96399999999999997},"min":{"1min":0.73799999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.581,"5min":2.5329999999999999,"15min":2.5329999999999999},"last":1.2749999999999999}]},{"osd":7,"last update":"Sun Sep 15 03:40:45 2024","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.018,"15min":0.92700000000000005},"min":{"1min":0.56599999999999995,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.498,"5min":3.238,"15min":3.238},"last":1.0269999999999999},{"interface":"front","average":{"1min":1.244,"5min":1.1299999999999999,"15min":1.109},"min":{"1min":0.68700000000000006,"5min":0.378,"15min":0.378},"max":{"1min":1.552,"5min":3.8260000000000001,"15min":3.8260000000000001},"last":1.226}]}]},{"osd":1,"up_from":13,"seq":55834574900,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":986284,"kb_used_data":676452,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":93385556,"statfs":{"total":96636764160,"available":95626809344,"internally_reserved":0,"allocated":692686848,"data_stored":692165832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 03:39:44 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.56100000000000005,"15min":0.54300000000000004},"min":{"1min":0.28699999999999998,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.94599999999999995,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.64200000000000002,"15min":0.59899999999999998},"min":{"1min":0.25700000000000001,"5min":0.187,"15min":0.187},"max":{"1min":6.6050000000000004,"5min":6.6050000000000004,"15min":6.6050000000000004},"last":0.77300000000000002}]},{"osd":2,"last update":"Sun Sep 15 03:39:44 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.621,"15min":0.625},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.171,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.746,"15min":0.73999999999999999},"min":{"1min":0.32300000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.2070000000000001,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":1.0580000000000001}]},{"osd":3,"last update":"Sun Sep 15 03:39:44 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.66800000000000004,"15min":0.63300000000000001},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.93200000000000005},{"interface":"front","average":{"1min":1.1459999999999999,"5min":0.80400000000000005,"15min":0.72499999999999998},"min":{"1min":0.59199999999999997,"5min":0.255,"15min":0.255},"max":{"1min":5.6950000000000003,"5min":5.6950000000000003,"15min":5.6950000000000003},"last":1.123}]},{"osd":4,"last update":"Sun Sep 15 03:39:44 2024","interfaces":[{"interface":"back","average":{"1min":1.498,"5min":0.99399999999999999,"15min":0.88800000000000001},"min":{"1min":0.38300000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":9.1080000000000005,"5min":9.1080000000000005,"15min":9.1080000000000005},"last":1.2729999999999999},{"interface":"front","average":{"1min":1.5980000000000001,"5min":1.0669999999999999,"15min":0.97299999999999998},"min":{"1min":0.53100000000000003,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":9.1440000000000001,"5min":9.1440000000000001,"15min":9.1440000000000001},"last":1.3740000000000001}]},{"osd":5,"last update":"Sun Sep 15 03:39:44 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.90700000000000003,"15min":0.89100000000000001},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.236,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.046,"5min":0.98899999999999999,"15min":0.96199999999999997},"min":{"1min":0.47699999999999998,"5min":0.314,"15min":0.314},"max":{"1min":1.4970000000000001,"5min":2.3580000000000001,"15min":2.3580000000000001},"last":1.4530000000000001}]},{"osd":6,"last update":"Sun Sep 15 03:39:44 2024","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":0.96899999999999997,"15min":0.92900000000000005},"min":{"1min":0.40699999999999997,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":6.4109999999999996,"5min":6.4109999999999996,"15min":6.4109999999999996},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.0289999999999999,"15min":0.999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":5.3170000000000002,"5min":5.3170000000000002,"15min":5.3170000000000002},"last":1.341}]},{"osd":7,"last update":"Sun Sep 15 03:39:44 2024","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.3340000000000001,"15min":1.45},"min":{"1min":0.46300000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":6.3730000000000002,"5min":12.821,"15min":12.821},"last":1.403},{"interface":"front","average":{"1min":1.222,"5min":1.089,"15min":1.0669999999999999},"min":{"1min":0.505,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":5.2530000000000001,"5min":5.2530000000000001,"15min":5.2530000000000001},"last":1.4890000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574901,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1362492,"kb_used_data":1056820,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":93009348,"statfs":{"total":96636764160,"available":95241572352,"internally_reserved":0,"allocated":1082183680,"data_stored":1081273737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Sep 15 03:40:41 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.59399999999999997,"15min":0.59999999999999998},"min":{"1min":0.20799999999999999,"5min":0.19,"15min":0.19},"max":{"1min":2.9279999999999999,"5min":2.9279999999999999,"15min":2.9279999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.58099999999999996,"15min":0.54400000000000004},"min":{"1min":0.41399999999999998,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.79500000000000004}]},{"osd":2,"last update":"Sun Sep 15 03:39:46 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.57499999999999996,"15min":0.54500000000000004},"min":{"1min":0.317,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.69499999999999995,"15min":0.69899999999999995},"min":{"1min":0.313,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.1839999999999999,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":0.93899999999999995}]},{"osd":3,"last update":"Sun Sep 15 03:39:46 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.70199999999999996,"15min":0.71799999999999997},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.135,"5min":1.603,"15min":1.603},"last":0.747},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.66700000000000004,"15min":0.64800000000000002},"min":{"1min":0.28499999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.1060000000000001,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.872}]},{"osd":4,"last update":"Sun Sep 15 03:40:41 2024","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.83099999999999996,"15min":0.80200000000000005},"min":{"1min":0.45900000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.1919999999999999,"5min":2.403,"15min":2.403},"last":1.036},{"interface":"front","average":{"1min":1.135,"5min":0.89700000000000002,"15min":0.83499999999999996},"min":{"1min":0.41299999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":3.1059999999999999,"5min":3.1059999999999999,"15min":3.1059999999999999},"last":1.0920000000000001}]},{"osd":5,"last update":"Sun Sep 15 03:39:46 2024","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.85399999999999998,"15min":0.81799999999999995},"min":{"1min":0.38100000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":1.002},{"interface":"front","average":{"1min":1.0169999999999999,"5min":0.91500000000000004,"15min":0.86699999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.5189999999999999,"5min":2.4319999999999999,"15min":2.4319999999999999},"last":1.3440000000000001}]},{"osd":6,"last update":"Sun Sep 15 03:39:46 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.88400000000000001,"15min":0.85799999999999998},"min":{"1min":0.53900000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.4550000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.02,"5min":1.006,"15min":1},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.425,"5min":2.5459999999999998,"15min":2.5459999999999998},"last":1.2689999999999999}]},{"osd":7,"last update":"Sun Sep 15 03:39:46 2024","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.242,"15min":1.3109999999999999},"min":{"1min":0.69699999999999995,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.4790000000000001,"5min":10.561999999999999,"15min":10.561999999999999},"last":1.179},{"interface":"front","average":{"1min":0.995,"5min":0.91300000000000003,"15min":0.88600000000000001},"min":{"1min":0.45700000000000002,"5min":0.377,"15min":0.377},"max":{"1min":1.4510000000000001,"5min":1.554,"15min":1.554},"last":1.1559999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1081413632,"data_stored":1080658160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":692445184,"data_stored":692079164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":386224128,"data_stored":386004840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1005244416,"data_stored":1004567120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1352245248,"data_stored":1351444108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":894988288,"data_stored":894396720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":357982208,"data_stored":357742880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1072160768,"data_stored":1071536856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T03:40:49.406 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T03:40:49.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.507+0000 7fabc4e63700 1 Processor -- start 2024-09-15T03:40:49.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.507+0000 7fabc4e63700 1 -- start start 2024-09-15T03:40:49.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.507+0000 7fabc4e63700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fabc013a2b0 0x7fabc00acd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:49.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.507+0000 7fabc4e63700 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabc00a50a0 con 0x7fabc00b1830 2024-09-15T03:40:49.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.507+0000 7fabc4e63700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabc013d7c0 con 0x7fabc00b1460 2024-09-15T03:40:49.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.507+0000 7fabc4e63700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabc013daa0 con 0x7fabc013a2b0 2024-09-15T03:40:49.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.507+0000 7fabbeffd700 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fabc00b1830 0x7fabc006ef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:32854/0 (socket says 172.21.15.110:32854) 2024-09-15T03:40:49.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.507+0000 7fabbeffd700 1 -- 172.21.15.110:0/4088710794 learned_addr learned my addr 172.21.15.110:0/4088710794 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-15T03:40:49.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.507+0000 7fabbffff700 1 --2- 172.21.15.110:0/4088710794 >> v2:172.21.15.177:3300/0 conn(0x7fabc013a2b0 0x7fabc00acd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:49.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.507+0000 7fabbe7fc700 1 -- 172.21.15.110:0/4088710794 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 943102719 0 0) 0x7fabc00a50a0 con 0x7fabc00b1830 2024-09-15T03:40:49.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.507+0000 7fabbe7fc700 1 -- 172.21.15.110:0/4088710794 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabac003570 con 0x7fabc00b1830 2024-09-15T03:40:49.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.507+0000 7fabbe7fc700 1 -- 172.21.15.110:0/4088710794 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 977478095 0 0) 0x7fabc013d7c0 con 0x7fabc00b1460 2024-09-15T03:40:49.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.507+0000 7fabbe7fc700 1 -- 172.21.15.110:0/4088710794 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabc00a50a0 con 0x7fabc00b1460 2024-09-15T03:40:49.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.507+0000 7fabbffff700 1 -- 172.21.15.110:0/4088710794 >> v1:172.21.15.177:6789/0 conn(0x7fabc00b1460 legacy=0x7fabc013df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:49.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.507+0000 7fabbffff700 1 -- 172.21.15.110:0/4088710794 >> v1:172.21.15.110:6789/0 conn(0x7fabc00b1830 legacy=0x7fabc006ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:49.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.507+0000 7fabbffff700 1 -- 172.21.15.110:0/4088710794 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc013e640 con 0x7fabc013a2b0 2024-09-15T03:40:49.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.507+0000 7fabbffff700 1 --2- 172.21.15.110:0/4088710794 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fabc013a2b0 0x7fabc00acd70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fabb80098e0 tx=0x7fabb80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32a3498a31d59721 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:49.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.511+0000 7fabbe7fc700 1 -- 172.21.15.110:0/4088710794 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb8011630 con 0x7fabc013a2b0 2024-09-15T03:40:49.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.511+0000 7fabbe7fc700 1 -- 172.21.15.110:0/4088710794 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabb8011790 con 0x7fabc013a2b0 2024-09-15T03:40:49.514 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.511+0000 7fabbe7fc700 1 -- 172.21.15.110:0/4088710794 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb8011b10 con 0x7fabc013a2b0 2024-09-15T03:40:49.515 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.511+0000 7fabc4e63700 1 -- 172.21.15.110:0/4088710794 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fabc013a2b0 msgr2=0x7fabc00acd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:49.515 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.511+0000 7fabc4e63700 1 --2- 172.21.15.110:0/4088710794 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fabc013a2b0 0x7fabc00acd70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fabb80098e0 tx=0x7fabb80091a0 comp rx=0 tx=0).stop 2024-09-15T03:40:49.515 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.511+0000 7fabc4e63700 1 -- 172.21.15.110:0/4088710794 shutdown_connections 2024-09-15T03:40:49.515 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.511+0000 7fabc4e63700 1 --2- 172.21.15.110:0/4088710794 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fabc013a2b0 0x7fabc00acd70 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:49.515 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.511+0000 7fabc4e63700 1 -- 172.21.15.110:0/4088710794 >> 172.21.15.110:0/4088710794 conn(0x7fabc0069130 msgr2=0x7fabc00b0d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:40:49.515 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.511+0000 7fabc4e63700 1 -- 172.21.15.110:0/4088710794 shutdown_connections 2024-09-15T03:40:49.515 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.511+0000 7fabc4e63700 1 -- 172.21.15.110:0/4088710794 wait complete. 2024-09-15T03:40:49.516 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.511+0000 7fabc4e63700 1 Processor -- start 2024-09-15T03:40:49.516 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.511+0000 7fabc4e63700 1 -- start start 2024-09-15T03:40:49.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.515+0000 7fabc4e63700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fabc00b1460 0x7fabc006c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:49.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.515+0000 7fabc4e63700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fabc00b1830 0x7fabc00aa510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:49.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.515+0000 7fabc4e63700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fabc013a2b0 0x7fabc00aaed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:49.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.515+0000 7fabc4e63700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc00aa080 con 0x7fabc00b1460 2024-09-15T03:40:49.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.515+0000 7fabc4e63700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc00aaa50 con 0x7fabc013a2b0 2024-09-15T03:40:49.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.515+0000 7fabc4e63700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabc00ac850 con 0x7fabc00b1830 2024-09-15T03:40:49.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.515+0000 7fabbf7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fabc00b1460 0x7fabc006c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:49.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.515+0000 7fabbeffd700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fabc00b1830 0x7fabc00aa510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:49.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.515+0000 7fabbf7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fabc00b1460 0x7fabc006c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51050/0 (socket says 172.21.15.110:51050) 2024-09-15T03:40:49.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.515+0000 7fabbf7fe700 1 -- 172.21.15.110:0/3951925778 learned_addr learned my addr 172.21.15.110:0/3951925778 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:40:49.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.515+0000 7fabbffff700 1 --2- 172.21.15.110:0/3951925778 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fabc013a2b0 0x7fabc00aaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:49.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.515+0000 7fabbf7fe700 1 -- 172.21.15.110:0/3951925778 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fabc00b1830 msgr2=0x7fabc00aa510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:49.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.515+0000 7fabbf7fe700 1 --2- 172.21.15.110:0/3951925778 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fabc00b1830 0x7fabc00aa510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:49.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.515+0000 7fabbf7fe700 1 -- 172.21.15.110:0/3951925778 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fabc013a2b0 msgr2=0x7fabc00aaed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:49.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.515+0000 7fabbf7fe700 1 --2- 172.21.15.110:0/3951925778 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fabc013a2b0 0x7fabc00aaed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:49.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.515+0000 7fabbf7fe700 1 -- 172.21.15.110:0/3951925778 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb8009420 con 0x7fabc00b1460 2024-09-15T03:40:49.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.515+0000 7fabbf7fe700 1 --2- 172.21.15.110:0/3951925778 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fabc00b1460 0x7fabc006c5f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7faba800c7c0 tx=0x7faba8007dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:49.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.515+0000 7fabb77fe700 1 -- 172.21.15.110:0/3951925778 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba8002ea0 con 0x7fabc00b1460 2024-09-15T03:40:49.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.515+0000 7fabb77fe700 1 -- 172.21.15.110:0/3951925778 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faba8010510 con 0x7fabc00b1460 2024-09-15T03:40:49.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.515+0000 7fabc4e63700 1 -- 172.21.15.110:0/3951925778 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabc01388e0 con 0x7fabc00b1460 2024-09-15T03:40:49.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.515+0000 7fabb77fe700 1 -- 172.21.15.110:0/3951925778 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba80108b0 con 0x7fabc00b1460 2024-09-15T03:40:49.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.515+0000 7fabc4e63700 1 -- 172.21.15.110:0/3951925778 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabc00a89f0 con 0x7fabc00b1460 2024-09-15T03:40:49.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.515+0000 7fabc4e63700 1 -- 172.21.15.110:0/3951925778 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabc00721b0 con 0x7fabc00b1460 2024-09-15T03:40:49.523 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.519+0000 7fabb77fe700 1 -- 172.21.15.110:0/3951925778 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) v1 ==== 85847+0+0 (secure 0 0 0) 0x7faba8010c20 con 0x7fabc00b1460 2024-09-15T03:40:49.523 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.519+0000 7fabb77fe700 1 --2- 172.21.15.110:0/3951925778 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7faba0067ab0 0x7faba0069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:49.523 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.519+0000 7fabb77fe700 1 -- 172.21.15.110:0/3951925778 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5641+0+0 (secure 0 0 0) 0x7faba8084d30 con 0x7fabc00b1460 2024-09-15T03:40:49.523 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.519+0000 7fabbeffd700 1 --2- 172.21.15.110:0/3951925778 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7faba0067ab0 0x7faba0069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:49.523 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.519+0000 7fabbeffd700 1 --2- 172.21.15.110:0/3951925778 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7faba0067ab0 0x7faba0069f60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fabc006cc50 tx=0x7fabb0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:49.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.527+0000 7fabb77fe700 1 -- 172.21.15.110:0/3951925778 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7faba8051e70 con 0x7fabc00b1460 2024-09-15T03:40:49.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.687+0000 7fabc4e63700 1 -- 172.21.15.110:0/3951925778 --> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fabc013d560 con 0x7faba0067ab0 2024-09-15T03:40:49.703 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.699+0000 7fabb77fe700 1 -- 172.21.15.110:0/3951925778 <== mgr.4104 v2:172.21.15.110:6832/22190 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+153374 (secure 0 0 0) 0x7fabc013d560 con 0x7faba0067ab0 2024-09-15T03:40:49.705 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-15T03:40:49.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.703+0000 7fabc4e63700 1 -- 172.21.15.110:0/3951925778 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7faba0067ab0 msgr2=0x7faba0069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:49.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.703+0000 7fabc4e63700 1 --2- 172.21.15.110:0/3951925778 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7faba0067ab0 0x7faba0069f60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fabc006cc50 tx=0x7fabb0009040 comp rx=0 tx=0).stop 2024-09-15T03:40:49.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.703+0000 7fabc4e63700 1 -- 172.21.15.110:0/3951925778 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fabc00b1460 msgr2=0x7fabc006c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:49.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.703+0000 7fabc4e63700 1 --2- 172.21.15.110:0/3951925778 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fabc00b1460 0x7fabc006c5f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7faba800c7c0 tx=0x7faba8007dc0 comp rx=0 tx=0).stop 2024-09-15T03:40:49.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.703+0000 7fabc4e63700 1 -- 172.21.15.110:0/3951925778 shutdown_connections 2024-09-15T03:40:49.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.703+0000 7fabc4e63700 1 --2- 172.21.15.110:0/3951925778 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7faba0067ab0 0x7faba0069f60 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:49.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.703+0000 7fabc4e63700 1 --2- 172.21.15.110:0/3951925778 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fabc013a2b0 0x7fabc00aaed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:49.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.703+0000 7fabc4e63700 1 --2- 172.21.15.110:0/3951925778 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fabc00b1830 0x7fabc00aa510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:49.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.703+0000 7fabc4e63700 1 --2- 172.21.15.110:0/3951925778 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fabc00b1460 0x7fabc006c5f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:49.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.703+0000 7fabc4e63700 1 -- 172.21.15.110:0/3951925778 >> 172.21.15.110:0/3951925778 conn(0x7fabc0069130 msgr2=0x7fabc006e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:40:49.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.703+0000 7fabc4e63700 1 -- 172.21.15.110:0/3951925778 shutdown_connections 2024-09-15T03:40:49.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.703+0000 7fabc4e63700 1 -- 172.21.15.110:0/3951925778 wait complete. 2024-09-15T03:40:49.708 INFO:teuthology.orchestra.run.smithi110.stderr:dumped all 2024-09-15T03:40:49.720 INFO:teuthology.orchestra.run.smithi110.stdout:{"pg_ready":true,"pg_map":{"version":143,"stamp":"2024-09-15T03:40:48.676511+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3570763067,"num_objects":883,"num_object_clones":0,"num_object_copies":1766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":883,"num_whiteouts":0,"num_read":2563,"num_read_kb":2563,"num_write":159105,"num_write_kb":5635359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160810,"ondisk_log_size":160810,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":9118160,"kb_used_data":6685256,"kb_used_omap":13,"kb_used_meta":2432818,"kb_avail":745856560,"statfs":{"total":773094113280,"available":763757117440,"internally_reserved":0,"allocated":6845702144,"data_stored":6840192686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2491206336},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003442"},"pg_stats":[{"pgid":"3.1f","version":"22'3025","reported_seq":3035,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T03:37:55.907201+0000","last_change":"2024-09-15T03:36:48.151935+0000","last_active":"2024-09-15T03:37:55.907201+0000","last_peered":"2024-09-15T03:37:55.907201+0000","last_clean":"2024-09-15T03:37:55.907201+0000","last_became_active":"2024-09-15T03:36:48.151625+0000","last_became_peered":"2024-09-15T03:36:48.151625+0000","last_unstale":"2024-09-15T03:37:55.907201+0000","last_undegraded":"2024-09-15T03:37:55.907201+0000","last_fullsized":"2024-09-15T03:37:55.907201+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":3025,"log_dups_size":0,"ondisk_log_size":3025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:02:31.103276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663296,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3019,"num_write_kb":136067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"23'6207","reported_seq":6522,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:40.006171+0000","last_change":"2024-09-15T03:36:48.150055+0000","last_active":"2024-09-15T03:39:40.006171+0000","last_peered":"2024-09-15T03:39:40.006171+0000","last_clean":"2024-09-15T03:39:40.006171+0000","last_became_active":"2024-09-15T03:36:48.149731+0000","last_became_peered":"2024-09-15T03:36:48.149731+0000","last_unstale":"2024-09-15T03:39:40.006171+0000","last_undegraded":"2024-09-15T03:39:40.006171+0000","last_fullsized":"2024-09-15T03:39:40.006171+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":6207,"log_dups_size":0,"ondisk_log_size":6207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:50:59.926991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113377280,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":1022,"num_read_kb":916,"num_write":6394,"num_write_kb":178051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"23'6287","reported_seq":6400,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:46.098160+0000","last_change":"2024-09-15T03:36:48.162431+0000","last_active":"2024-09-15T03:39:46.098160+0000","last_peered":"2024-09-15T03:39:46.098160+0000","last_clean":"2024-09-15T03:39:46.098160+0000","last_became_active":"2024-09-15T03:36:48.162085+0000","last_became_peered":"2024-09-15T03:36:48.162085+0000","last_unstale":"2024-09-15T03:39:46.098160+0000","last_undegraded":"2024-09-15T03:39:46.098160+0000","last_fullsized":"2024-09-15T03:39:46.098160+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":6287,"log_dups_size":0,"ondisk_log_size":6287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:51:54.108247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121898016,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":109,"num_read_kb":107,"num_write":6275,"num_write_kb":177699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"22'4656","reported_seq":4674,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T03:37:56.481330+0000","last_change":"2024-09-15T03:36:48.159640+0000","last_active":"2024-09-15T03:37:56.481330+0000","last_peered":"2024-09-15T03:37:56.481330+0000","last_clean":"2024-09-15T03:37:56.481330+0000","last_became_active":"2024-09-15T03:36:48.159272+0000","last_became_peered":"2024-09-15T03:36:48.159272+0000","last_unstale":"2024-09-15T03:37:56.481330+0000","last_undegraded":"2024-09-15T03:37:56.481330+0000","last_fullsized":"2024-09-15T03:37:56.481330+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":4656,"log_dups_size":0,"ondisk_log_size":4656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:17:25.869833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88866816,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":4665,"num_write_kb":169740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"23'4796","reported_seq":4806,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:32.771695+0000","last_change":"2024-09-15T03:36:48.150332+0000","last_active":"2024-09-15T03:39:32.771695+0000","last_peered":"2024-09-15T03:39:32.771695+0000","last_clean":"2024-09-15T03:39:32.771695+0000","last_became_active":"2024-09-15T03:36:48.150057+0000","last_became_peered":"2024-09-15T03:36:48.150057+0000","last_unstale":"2024-09-15T03:39:32.771695+0000","last_undegraded":"2024-09-15T03:39:32.771695+0000","last_fullsized":"2024-09-15T03:39:32.771695+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":4796,"log_dups_size":0,"ondisk_log_size":4796,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:15:39.747133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117440512,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4750,"num_write_kb":185221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"23'4713","reported_seq":4728,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:38.053359+0000","last_change":"2024-09-15T03:36:48.144892+0000","last_active":"2024-09-15T03:39:38.053359+0000","last_peered":"2024-09-15T03:39:38.053359+0000","last_clean":"2024-09-15T03:39:38.053359+0000","last_became_active":"2024-09-15T03:36:48.144097+0000","last_became_peered":"2024-09-15T03:36:48.144097+0000","last_unstale":"2024-09-15T03:39:38.053359+0000","last_undegraded":"2024-09-15T03:39:38.053359+0000","last_fullsized":"2024-09-15T03:39:38.053359+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":4713,"log_dups_size":0,"ondisk_log_size":4713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:16:01.325609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68681728,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":22,"num_read_kb":16,"num_write":4699,"num_write_kb":143618,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"23'4491","reported_seq":4502,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:50.201809+0000","last_change":"2024-09-15T03:36:48.142399+0000","last_active":"2024-09-15T03:39:50.201809+0000","last_peered":"2024-09-15T03:39:50.201809+0000","last_clean":"2024-09-15T03:39:50.201809+0000","last_became_active":"2024-09-15T03:36:48.142085+0000","last_became_peered":"2024-09-15T03:36:48.142085+0000","last_unstale":"2024-09-15T03:39:50.201809+0000","last_undegraded":"2024-09-15T03:39:50.201809+0000","last_fullsized":"2024-09-15T03:39:50.201809+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":4491,"log_dups_size":0,"ondisk_log_size":4491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T13:00:10.524261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":111872344,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":4448,"num_write_kb":183583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"23'7205","reported_seq":7219,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:50.436933+0000","last_change":"2024-09-15T03:36:48.138540+0000","last_active":"2024-09-15T03:39:50.436933+0000","last_peered":"2024-09-15T03:39:50.436933+0000","last_clean":"2024-09-15T03:39:50.436933+0000","last_became_active":"2024-09-15T03:36:48.138049+0000","last_became_peered":"2024-09-15T03:36:48.138049+0000","last_unstale":"2024-09-15T03:39:50.436933+0000","last_undegraded":"2024-09-15T03:39:50.436933+0000","last_fullsized":"2024-09-15T03:39:50.436933+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":7205,"log_dups_size":0,"ondisk_log_size":7205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T12:37:15.684685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113246720,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":15,"num_read_kb":11,"num_write":7218,"num_write_kb":193736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"23'7862","reported_seq":7879,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:54.842187+0000","last_change":"2024-09-15T03:36:48.150100+0000","last_active":"2024-09-15T03:39:54.842187+0000","last_peered":"2024-09-15T03:39:54.842187+0000","last_clean":"2024-09-15T03:39:54.842187+0000","last_became_active":"2024-09-15T03:36:48.149792+0000","last_became_peered":"2024-09-15T03:36:48.149792+0000","last_unstale":"2024-09-15T03:39:54.842187+0000","last_undegraded":"2024-09-15T03:39:54.842187+0000","last_fullsized":"2024-09-15T03:39:54.842187+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":7862,"log_dups_size":0,"ondisk_log_size":7862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:42:07.633298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":146800640,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":7874,"num_write_kb":194640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"23'2408","reported_seq":2421,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:32.679416+0000","last_change":"2024-09-15T03:36:48.155132+0000","last_active":"2024-09-15T03:39:32.679416+0000","last_peered":"2024-09-15T03:39:32.679416+0000","last_clean":"2024-09-15T03:39:32.679416+0000","last_became_active":"2024-09-15T03:36:48.154800+0000","last_became_peered":"2024-09-15T03:36:48.154800+0000","last_unstale":"2024-09-15T03:39:32.679416+0000","last_undegraded":"2024-09-15T03:39:32.679416+0000","last_fullsized":"2024-09-15T03:39:32.679416+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":2408,"log_dups_size":0,"ondisk_log_size":2408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:54:21.705564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":2388,"num_write_kb":127362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"23'6301","reported_seq":6317,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:48.732682+0000","last_change":"2024-09-15T03:36:48.152499+0000","last_active":"2024-09-15T03:39:48.732682+0000","last_peered":"2024-09-15T03:39:48.732682+0000","last_clean":"2024-09-15T03:39:48.732682+0000","last_became_active":"2024-09-15T03:36:48.152191+0000","last_became_peered":"2024-09-15T03:36:48.152191+0000","last_unstale":"2024-09-15T03:39:48.732682+0000","last_undegraded":"2024-09-15T03:39:48.732682+0000","last_fullsized":"2024-09-15T03:39:48.732682+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":6301,"log_dups_size":0,"ondisk_log_size":6301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T12:12:56.810421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":124518928,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":19,"num_read_kb":14,"num_write":6316,"num_write_kb":168780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"23'5385","reported_seq":5398,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:42.445288+0000","last_change":"2024-09-15T03:36:48.154566+0000","last_active":"2024-09-15T03:39:42.445288+0000","last_peered":"2024-09-15T03:39:42.445288+0000","last_clean":"2024-09-15T03:39:42.445288+0000","last_became_active":"2024-09-15T03:36:48.153926+0000","last_became_peered":"2024-09-15T03:36:48.153926+0000","last_unstale":"2024-09-15T03:39:42.445288+0000","last_undegraded":"2024-09-15T03:39:42.445288+0000","last_fullsized":"2024-09-15T03:39:42.445288+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":5385,"log_dups_size":0,"ondisk_log_size":5385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:19:06.740804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109121016,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":5342,"num_write_kb":144586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:32.448890+0000","last_change":"2024-09-15T03:36:42.814645+0000","last_active":"2024-09-15T03:39:32.448890+0000","last_peered":"2024-09-15T03:39:32.448890+0000","last_clean":"2024-09-15T03:39:32.448890+0000","last_became_active":"2024-09-15T03:36:40.672828+0000","last_became_peered":"2024-09-15T03:36:40.672828+0000","last_unstale":"2024-09-15T03:39:32.448890+0000","last_undegraded":"2024-09-15T03:39:32.448890+0000","last_fullsized":"2024-09-15T03:39:32.448890+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_clean_scrub_stamp":"2024-09-15T03:36:39.639431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T13:40:53.039569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064069099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'5981","reported_seq":6014,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:54.327858+0000","last_change":"2024-09-15T03:36:48.143463+0000","last_active":"2024-09-15T03:39:54.327858+0000","last_peered":"2024-09-15T03:39:54.327858+0000","last_clean":"2024-09-15T03:39:54.327858+0000","last_became_active":"2024-09-15T03:36:48.143177+0000","last_became_peered":"2024-09-15T03:36:48.143177+0000","last_unstale":"2024-09-15T03:39:54.327858+0000","last_undegraded":"2024-09-15T03:39:54.327858+0000","last_fullsized":"2024-09-15T03:39:54.327858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":5981,"log_dups_size":0,"ondisk_log_size":5981,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:20:24.926833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":134639616,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":92,"num_read_kb":80,"num_write":5989,"num_write_kb":192351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T03:36:51.577473+0000","last_change":"2024-09-15T03:36:42.687267+0000","last_active":"2024-09-15T03:36:51.577473+0000","last_peered":"2024-09-15T03:36:51.577473+0000","last_clean":"2024-09-15T03:36:51.577473+0000","last_became_active":"2024-09-15T03:36:40.671677+0000","last_became_peered":"2024-09-15T03:36:40.671677+0000","last_unstale":"2024-09-15T03:36:51.577473+0000","last_undegraded":"2024-09-15T03:36:51.577473+0000","last_fullsized":"2024-09-15T03:36:51.577473+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_clean_scrub_stamp":"2024-09-15T03:36:39.639431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:55:15.493964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000861089,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"23'5400","reported_seq":5410,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:32.770837+0000","last_change":"2024-09-15T03:36:48.152124+0000","last_active":"2024-09-15T03:39:32.770837+0000","last_peered":"2024-09-15T03:39:32.770837+0000","last_clean":"2024-09-15T03:39:32.770837+0000","last_became_active":"2024-09-15T03:36:48.151844+0000","last_became_peered":"2024-09-15T03:36:48.151844+0000","last_unstale":"2024-09-15T03:39:32.770837+0000","last_undegraded":"2024-09-15T03:39:32.770837+0000","last_fullsized":"2024-09-15T03:39:32.770837+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":5400,"log_dups_size":0,"ondisk_log_size":5400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:17:16.444530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":115998720,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5352,"num_write_kb":174984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T03:36:51.579975+0000","last_change":"2024-09-15T03:36:42.854270+0000","last_active":"2024-09-15T03:36:51.579975+0000","last_peered":"2024-09-15T03:36:51.579975+0000","last_clean":"2024-09-15T03:36:51.579975+0000","last_became_active":"2024-09-15T03:36:41.079216+0000","last_became_peered":"2024-09-15T03:36:41.079216+0000","last_unstale":"2024-09-15T03:36:51.579975+0000","last_undegraded":"2024-09-15T03:36:51.579975+0000","last_fullsized":"2024-09-15T03:36:51.579975+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_clean_scrub_stamp":"2024-09-15T03:36:39.639431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:19:37.610607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062568200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"23'7270","reported_seq":7291,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:53.195871+0000","last_change":"2024-09-15T03:36:48.216882+0000","last_active":"2024-09-15T03:39:53.195871+0000","last_peered":"2024-09-15T03:39:53.195871+0000","last_clean":"2024-09-15T03:39:53.195871+0000","last_became_active":"2024-09-15T03:36:48.216337+0000","last_became_peered":"2024-09-15T03:36:48.216337+0000","last_unstale":"2024-09-15T03:39:53.195871+0000","last_undegraded":"2024-09-15T03:39:53.195871+0000","last_fullsized":"2024-09-15T03:39:53.195871+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":7270,"log_dups_size":0,"ondisk_log_size":7270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:27:25.000492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":146931712,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":45,"num_read_kb":34,"num_write":7284,"num_write_kb":246402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T03:36:51.577450+0000","last_change":"2024-09-15T03:36:42.687270+0000","last_active":"2024-09-15T03:36:51.577450+0000","last_peered":"2024-09-15T03:36:51.577450+0000","last_clean":"2024-09-15T03:36:51.577450+0000","last_became_active":"2024-09-15T03:36:41.077266+0000","last_became_peered":"2024-09-15T03:36:41.077266+0000","last_unstale":"2024-09-15T03:36:51.577450+0000","last_undegraded":"2024-09-15T03:36:51.577450+0000","last_fullsized":"2024-09-15T03:36:51.577450+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_clean_scrub_stamp":"2024-09-15T03:36:39.639431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:54:58.435053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067514400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"22'3282","reported_seq":3292,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T03:37:53.949949+0000","last_change":"2024-09-15T03:36:48.151841+0000","last_active":"2024-09-15T03:37:53.949949+0000","last_peered":"2024-09-15T03:37:53.949949+0000","last_clean":"2024-09-15T03:37:53.949949+0000","last_became_active":"2024-09-15T03:36:48.151520+0000","last_became_peered":"2024-09-15T03:36:48.151520+0000","last_unstale":"2024-09-15T03:37:53.949949+0000","last_undegraded":"2024-09-15T03:37:53.949949+0000","last_fullsized":"2024-09-15T03:37:53.949949+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":3282,"log_dups_size":0,"ondisk_log_size":3282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:24:47.104486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3278,"num_write_kb":144390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"22'3449","reported_seq":3473,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T03:37:58.123758+0000","last_change":"2024-09-15T03:36:48.146454+0000","last_active":"2024-09-15T03:37:58.123758+0000","last_peered":"2024-09-15T03:37:58.123758+0000","last_clean":"2024-09-15T03:37:58.123758+0000","last_became_active":"2024-09-15T03:36:48.145845+0000","last_became_peered":"2024-09-15T03:36:48.145845+0000","last_unstale":"2024-09-15T03:37:58.123758+0000","last_undegraded":"2024-09-15T03:37:58.123758+0000","last_fullsized":"2024-09-15T03:37:58.123758+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":3449,"log_dups_size":0,"ondisk_log_size":3449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:46:57.211180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113246208,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":50,"num_read_kb":41,"num_write":3479,"num_write_kb":144783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T03:36:50.383528+0000","last_change":"2024-09-15T03:36:42.695885+0000","last_active":"2024-09-15T03:36:50.383528+0000","last_peered":"2024-09-15T03:36:50.383528+0000","last_clean":"2024-09-15T03:36:50.383528+0000","last_became_active":"2024-09-15T03:36:40.673743+0000","last_became_peered":"2024-09-15T03:36:40.673743+0000","last_unstale":"2024-09-15T03:36:50.383528+0000","last_undegraded":"2024-09-15T03:36:50.383528+0000","last_fullsized":"2024-09-15T03:36:50.383528+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_clean_scrub_stamp":"2024-09-15T03:36:39.639431+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T13:57:31.462981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002159518,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"23'4710","reported_seq":4734,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:32.771799+0000","last_change":"2024-09-15T03:36:48.216887+0000","last_active":"2024-09-15T03:39:32.771799+0000","last_peered":"2024-09-15T03:39:32.771799+0000","last_clean":"2024-09-15T03:39:32.771799+0000","last_became_active":"2024-09-15T03:36:48.216269+0000","last_became_peered":"2024-09-15T03:36:48.216269+0000","last_unstale":"2024-09-15T03:39:32.771799+0000","last_undegraded":"2024-09-15T03:39:32.771799+0000","last_fullsized":"2024-09-15T03:39:32.771799+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":4710,"log_dups_size":0,"ondisk_log_size":4710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T12:10:11.804094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":142877200,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":51,"num_read_kb":164,"num_write":4698,"num_write_kb":182422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:32.402704+0000","last_change":"2024-09-15T03:36:42.801569+0000","last_active":"2024-09-15T03:39:32.402704+0000","last_peered":"2024-09-15T03:39:32.402704+0000","last_clean":"2024-09-15T03:39:32.402704+0000","last_became_active":"2024-09-15T03:36:40.669392+0000","last_became_peered":"2024-09-15T03:36:40.669392+0000","last_unstale":"2024-09-15T03:39:32.402704+0000","last_undegraded":"2024-09-15T03:39:32.402704+0000","last_fullsized":"2024-09-15T03:39:32.402704+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_clean_scrub_stamp":"2024-09-15T03:36:39.639431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:00:33.102146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059629799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"23'3227","reported_seq":3245,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:32.683131+0000","last_change":"2024-09-15T03:36:48.144585+0000","last_active":"2024-09-15T03:39:32.683131+0000","last_peered":"2024-09-15T03:39:32.683131+0000","last_clean":"2024-09-15T03:39:32.683131+0000","last_became_active":"2024-09-15T03:36:48.144313+0000","last_became_peered":"2024-09-15T03:36:48.144313+0000","last_unstale":"2024-09-15T03:39:32.683131+0000","last_undegraded":"2024-09-15T03:39:32.683131+0000","last_fullsized":"2024-09-15T03:39:32.683131+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":3227,"log_dups_size":0,"ondisk_log_size":3227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T15:27:39.715172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92405760,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":33,"num_read_kb":87,"num_write":3327,"num_write_kb":128260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T03:36:51.579392+0000","last_change":"2024-09-15T03:36:42.854222+0000","last_active":"2024-09-15T03:36:51.579392+0000","last_peered":"2024-09-15T03:36:51.579392+0000","last_clean":"2024-09-15T03:36:51.579392+0000","last_became_active":"2024-09-15T03:36:40.672416+0000","last_became_peered":"2024-09-15T03:36:40.672416+0000","last_unstale":"2024-09-15T03:36:51.579392+0000","last_undegraded":"2024-09-15T03:36:51.579392+0000","last_fullsized":"2024-09-15T03:36:51.579392+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_clean_scrub_stamp":"2024-09-15T03:36:39.639431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T15:08:21.420921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065187600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"23'3107","reported_seq":3117,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:46.717682+0000","last_change":"2024-09-15T03:36:48.151148+0000","last_active":"2024-09-15T03:39:46.717682+0000","last_peered":"2024-09-15T03:39:46.717682+0000","last_clean":"2024-09-15T03:39:46.717682+0000","last_became_active":"2024-09-15T03:36:48.150693+0000","last_became_peered":"2024-09-15T03:36:48.150693+0000","last_unstale":"2024-09-15T03:39:46.717682+0000","last_undegraded":"2024-09-15T03:39:46.717682+0000","last_fullsized":"2024-09-15T03:39:46.717682+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":3107,"log_dups_size":0,"ondisk_log_size":3107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:27:52.876606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67108864,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3105,"num_write_kb":103402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T03:36:50.383548+0000","last_change":"2024-09-15T03:36:42.694486+0000","last_active":"2024-09-15T03:36:50.383548+0000","last_peered":"2024-09-15T03:36:50.383548+0000","last_clean":"2024-09-15T03:36:50.383548+0000","last_became_active":"2024-09-15T03:36:40.674442+0000","last_became_peered":"2024-09-15T03:36:40.674442+0000","last_unstale":"2024-09-15T03:36:50.383548+0000","last_undegraded":"2024-09-15T03:36:50.383548+0000","last_fullsized":"2024-09-15T03:36:50.383548+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_clean_scrub_stamp":"2024-09-15T03:36:39.639431+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:25:57.474430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067001500000000004,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":106,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T03:36:51.579500+0000","last_change":"2024-09-15T03:36:38.815652+0000","last_active":"2024-09-15T03:36:51.579500+0000","last_peered":"2024-09-15T03:36:51.579500+0000","last_clean":"2024-09-15T03:36:51.579500+0000","last_became_active":"2024-09-15T03:36:38.813490+0000","last_became_peered":"2024-09-15T03:36:38.813490+0000","last_unstale":"2024-09-15T03:36:51.579500+0000","last_undegraded":"2024-09-15T03:36:51.579500+0000","last_fullsized":"2024-09-15T03:36:51.579500+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:37.618040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:37.618040+0000","last_clean_scrub_stamp":"2024-09-15T03:36:37.618040+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:02:12.095255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"23'5187","reported_seq":5201,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:32.773823+0000","last_change":"2024-09-15T03:36:48.142652+0000","last_active":"2024-09-15T03:39:32.773823+0000","last_peered":"2024-09-15T03:39:32.773823+0000","last_clean":"2024-09-15T03:39:32.773823+0000","last_became_active":"2024-09-15T03:36:48.142313+0000","last_became_peered":"2024-09-15T03:36:48.142313+0000","last_unstale":"2024-09-15T03:39:32.773823+0000","last_undegraded":"2024-09-15T03:39:32.773823+0000","last_fullsized":"2024-09-15T03:39:32.773823+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":5187,"log_dups_size":0,"ondisk_log_size":5187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T13:50:10.152068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":116760576,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":12,"num_read_kb":135,"num_write":5130,"num_write_kb":177200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"23'7935","reported_seq":8051,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:49.720540+0000","last_change":"2024-09-15T03:36:48.218591+0000","last_active":"2024-09-15T03:39:49.720540+0000","last_peered":"2024-09-15T03:39:49.720540+0000","last_clean":"2024-09-15T03:39:49.720540+0000","last_became_active":"2024-09-15T03:36:48.218083+0000","last_became_peered":"2024-09-15T03:36:48.218083+0000","last_unstale":"2024-09-15T03:39:49.720540+0000","last_undegraded":"2024-09-15T03:39:49.720540+0000","last_fullsized":"2024-09-15T03:39:49.720540+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":7935,"log_dups_size":0,"ondisk_log_size":7935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:15:32.571275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663808,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":356,"num_read_kb":379,"num_write":5975,"num_write_kb":152245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"24'5837","reported_seq":5868,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:48.324243+0000","last_change":"2024-09-15T03:36:48.155747+0000","last_active":"2024-09-15T03:40:48.324243+0000","last_peered":"2024-09-15T03:40:48.324243+0000","last_clean":"2024-09-15T03:40:48.324243+0000","last_became_active":"2024-09-15T03:36:48.155263+0000","last_became_peered":"2024-09-15T03:36:48.155263+0000","last_unstale":"2024-09-15T03:40:48.324243+0000","last_undegraded":"2024-09-15T03:40:48.324243+0000","last_fullsized":"2024-09-15T03:40:48.324243+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":5837,"log_dups_size":0,"ondisk_log_size":5837,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:40:42.479597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":137103712,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":68,"num_read_kb":57,"num_write":5841,"num_write_kb":205281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"22'3150","reported_seq":3160,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T03:37:55.878205+0000","last_change":"2024-09-15T03:36:48.138000+0000","last_active":"2024-09-15T03:37:55.878205+0000","last_peered":"2024-09-15T03:37:55.878205+0000","last_clean":"2024-09-15T03:37:55.878205+0000","last_became_active":"2024-09-15T03:36:48.137589+0000","last_became_peered":"2024-09-15T03:36:48.137589+0000","last_unstale":"2024-09-15T03:37:55.878205+0000","last_undegraded":"2024-09-15T03:37:55.878205+0000","last_fullsized":"2024-09-15T03:37:55.878205+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":3150,"log_dups_size":0,"ondisk_log_size":3150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:02:31.208417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3149,"num_write_kb":152324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"23'4730","reported_seq":4757,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:32.766647+0000","last_change":"2024-09-15T03:36:48.151666+0000","last_active":"2024-09-15T03:39:32.766647+0000","last_peered":"2024-09-15T03:39:32.766647+0000","last_clean":"2024-09-15T03:39:32.766647+0000","last_became_active":"2024-09-15T03:36:48.151365+0000","last_became_peered":"2024-09-15T03:36:48.151365+0000","last_unstale":"2024-09-15T03:39:32.766647+0000","last_undegraded":"2024-09-15T03:39:32.766647+0000","last_fullsized":"2024-09-15T03:39:32.766647+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":4730,"log_dups_size":0,"ondisk_log_size":4730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T15:05:31.496486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121635328,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":72,"num_read_kb":60,"num_write":4709,"num_write_kb":215880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"23'5517","reported_seq":5531,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:32.678901+0000","last_change":"2024-09-15T03:36:48.155332+0000","last_active":"2024-09-15T03:39:32.678901+0000","last_peered":"2024-09-15T03:39:32.678901+0000","last_clean":"2024-09-15T03:39:32.678901+0000","last_became_active":"2024-09-15T03:36:48.154955+0000","last_became_peered":"2024-09-15T03:36:48.154955+0000","last_unstale":"2024-09-15T03:39:32.678901+0000","last_undegraded":"2024-09-15T03:39:32.678901+0000","last_fullsized":"2024-09-15T03:39:32.678901+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":5517,"log_dups_size":0,"ondisk_log_size":5517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:17:29.191405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117440512,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":5499,"num_write_kb":248958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"23'4086","reported_seq":4166,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:32.831744+0000","last_change":"2024-09-15T03:36:48.162708+0000","last_active":"2024-09-15T03:39:32.831744+0000","last_peered":"2024-09-15T03:39:32.831744+0000","last_clean":"2024-09-15T03:39:32.831744+0000","last_became_active":"2024-09-15T03:36:48.162401+0000","last_became_peered":"2024-09-15T03:36:48.162401+0000","last_unstale":"2024-09-15T03:39:32.831744+0000","last_undegraded":"2024-09-15T03:39:32.831744+0000","last_fullsized":"2024-09-15T03:39:32.831744+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":4086,"log_dups_size":0,"ondisk_log_size":4086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:15:25.156037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104866320,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":269,"num_read_kb":201,"num_write":4343,"num_write_kb":177773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"23'3614","reported_seq":3643,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:32.772767+0000","last_change":"2024-09-15T03:36:48.162180+0000","last_active":"2024-09-15T03:39:32.772767+0000","last_peered":"2024-09-15T03:39:32.772767+0000","last_clean":"2024-09-15T03:39:32.772767+0000","last_became_active":"2024-09-15T03:36:48.161897+0000","last_became_peered":"2024-09-15T03:36:48.161897+0000","last_unstale":"2024-09-15T03:39:32.772767+0000","last_undegraded":"2024-09-15T03:39:32.772767+0000","last_fullsized":"2024-09-15T03:39:32.772767+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":3614,"log_dups_size":0,"ondisk_log_size":3614,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:44:27.564730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":130023424,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":75,"num_read_kb":60,"num_write":3605,"num_write_kb":185220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"24'4055","reported_seq":4066,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:48.332171+0000","last_change":"2024-09-15T03:36:48.149972+0000","last_active":"2024-09-15T03:40:48.332171+0000","last_peered":"2024-09-15T03:40:48.332171+0000","last_clean":"2024-09-15T03:40:48.332171+0000","last_became_active":"2024-09-15T03:36:48.149651+0000","last_became_peered":"2024-09-15T03:36:48.149651+0000","last_unstale":"2024-09-15T03:40:48.332171+0000","last_undegraded":"2024-09-15T03:40:48.332171+0000","last_fullsized":"2024-09-15T03:40:48.332171+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":4055,"log_dups_size":0,"ondisk_log_size":4055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T15:32:38.282915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":111869952,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4006,"num_write_kb":198851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"22'2797","reported_seq":2823,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T03:37:53.771805+0000","last_change":"2024-09-15T03:36:48.159682+0000","last_active":"2024-09-15T03:37:53.771805+0000","last_peered":"2024-09-15T03:37:53.771805+0000","last_clean":"2024-09-15T03:37:53.771805+0000","last_became_active":"2024-09-15T03:36:48.159380+0000","last_became_peered":"2024-09-15T03:36:48.159380+0000","last_unstale":"2024-09-15T03:37:53.771805+0000","last_undegraded":"2024-09-15T03:37:53.771805+0000","last_fullsized":"2024-09-15T03:37:53.771805+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":2797,"log_dups_size":0,"ondisk_log_size":2797,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T13:43:53.575032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":50,"num_read_kb":41,"num_write":2819,"num_write_kb":144138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"23'5712","reported_seq":5727,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:32.775461+0000","last_change":"2024-09-15T03:36:48.151085+0000","last_active":"2024-09-15T03:39:32.775461+0000","last_peered":"2024-09-15T03:39:32.775461+0000","last_clean":"2024-09-15T03:39:32.775461+0000","last_became_active":"2024-09-15T03:36:48.150784+0000","last_became_peered":"2024-09-15T03:36:48.150784+0000","last_unstale":"2024-09-15T03:39:32.775461+0000","last_undegraded":"2024-09-15T03:39:32.775461+0000","last_fullsized":"2024-09-15T03:39:32.775461+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":5712,"log_dups_size":0,"ondisk_log_size":5712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:11:11.620798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":138543104,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":22,"num_read_kb":16,"num_write":5677,"num_write_kb":255624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"23'8353","reported_seq":8372,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:46.094057+0000","last_change":"2024-09-15T03:36:48.159758+0000","last_active":"2024-09-15T03:39:46.094057+0000","last_peered":"2024-09-15T03:39:46.094057+0000","last_clean":"2024-09-15T03:39:46.094057+0000","last_became_active":"2024-09-15T03:36:48.159470+0000","last_became_peered":"2024-09-15T03:36:48.159470+0000","last_unstale":"2024-09-15T03:39:46.094057+0000","last_undegraded":"2024-09-15T03:39:46.094057+0000","last_fullsized":"2024-09-15T03:39:46.094057+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":8353,"log_dups_size":0,"ondisk_log_size":8353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:06:20.271144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":126026248,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":36,"num_read_kb":28,"num_write":8349,"num_write_kb":200158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":3570172680,"num_objects":879,"num_object_clones":0,"num_object_copies":1758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":879,"num_whiteouts":0,"num_read":2497,"num_read_kb":2508,"num_write":159002,"num_write_kb":5633729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":6842703872,"data_stored":6838429848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160735,"ondisk_log_size":160735,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542196,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1357616,"kb_used_data":1047784,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":93014224,"statfs":{"total":96636764160,"available":95246565376,"internally_reserved":0,"allocated":1072930816,"data_stored":1072152433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 03:39:47 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":2.0049999999999999,"15min":2.5830000000000002},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.1799999999999999,"5min":35.189,"15min":35.189},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.749,"15min":0.74199999999999999},"min":{"1min":0.39900000000000002,"5min":0.245,"15min":0.245},"max":{"1min":1.173,"5min":1.427,"15min":1.427},"last":0.76500000000000001}]},{"osd":1,"last update":"Sun Sep 15 03:39:47 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":2.0150000000000001,"15min":2.544},"min":{"1min":0.315,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.54,"5min":35.223999999999997,"15min":35.223999999999997},"last":1.1779999999999999},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78700000000000003,"15min":0.76700000000000002},"min":{"1min":0.38300000000000001,"5min":0.308,"15min":0.308},"max":{"1min":1.1930000000000001,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.84799999999999998}]},{"osd":2,"last update":"Sun Sep 15 03:39:47 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.73199999999999998,"15min":0.72899999999999998},"min":{"1min":0.374,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.2509999999999999,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":1.0409999999999999},{"interface":"front","average":{"1min":0.81799999999999995,"5min":2.105,"15min":2.641},"min":{"1min":0.40899999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.3879999999999999,"5min":35.283000000000001,"15min":35.283000000000001},"last":1.0149999999999999}]},{"osd":3,"last update":"Sun Sep 15 03:39:47 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.78400000000000003,"15min":0.74099999999999999},"min":{"1min":0.45500000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.266,"5min":1.71,"15min":1.71},"last":1.117},{"interface":"front","average":{"1min":0.92000000000000004,"5min":2.1749999999999998,"15min":2.6930000000000001},"min":{"1min":0.50800000000000001,"5min":0.378,"15min":0.378},"max":{"1min":1.3580000000000001,"5min":35.255000000000003,"15min":35.255000000000003},"last":0.97999999999999998}]},{"osd":4,"last update":"Sun Sep 15 03:39:47 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":1.048,"15min":0.80800000000000005},"min":{"1min":0.33900000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.3380000000000001,"5min":25.52,"15min":25.52},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.81699999999999995,"15min":0.77300000000000002},"min":{"1min":0.32200000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.339,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":1.1499999999999999}]},{"osd":5,"last update":"Sun Sep 15 03:39:47 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":2.238,"15min":2.778},"min":{"1min":0.41599999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.329,"5min":35.070999999999998,"15min":35.070999999999998},"last":1.2290000000000001},{"interface":"front","average":{"1min":0.96999999999999997,"5min":1.0820000000000001,"15min":1.101},"min":{"1min":0.43099999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.4430000000000001,"5min":5.2999999999999998,"15min":5.2999999999999998},"last":1.2}]},{"osd":6,"last update":"Sun Sep 15 03:39:47 2024","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.94399999999999995,"15min":0.878},"min":{"1min":0.42099999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.355,"5min":2.5449999999999999,"15min":2.5449999999999999},"last":1.085},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.90700000000000003,"15min":0.84699999999999998},"min":{"1min":0.40799999999999997,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.484,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":1.452}]}]},{"osd":6,"up_from":14,"seq":60129542196,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":651352,"kb_used_data":349836,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93720488,"statfs":{"total":96636764160,"available":95969779712,"internally_reserved":0,"allocated":358232064,"data_stored":357835405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 03:40:43 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":1.0049999999999999,"15min":1.155},"min":{"1min":0.253,"5min":0.25,"15min":0.25},"max":{"1min":1.208,"5min":7.3170000000000002,"15min":7.3170000000000002},"last":1.3069999999999999},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.82399999999999995,"15min":0.77300000000000002},"min":{"1min":0.47799999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.212,"5min":7.3849999999999998,"15min":7.3849999999999998},"last":1.157}]},{"osd":1,"last update":"Sun Sep 15 03:40:43 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.72899999999999998,"15min":0.69099999999999995},"min":{"1min":0.30299999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.157,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.89600000000000002,"15min":0.91800000000000004},"min":{"1min":0.45800000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.304,"5min":7.3559999999999999,"15min":7.3559999999999999},"last":1.1100000000000001}]},{"osd":2,"last update":"Sun Sep 15 03:40:43 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.92800000000000005,"15min":0.93500000000000005},"min":{"1min":0.436,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.286,"5min":7.4960000000000004,"15min":7.4960000000000004},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.84699999999999998,"15min":0.88600000000000001},"min":{"1min":0.49199999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.1499999999999999,"5min":1.575,"15min":1.575},"last":1.474}]},{"osd":3,"last update":"Sun Sep 15 03:40:43 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":1.0569999999999999,"15min":1.1499999999999999},"min":{"1min":0.42399999999999999,"5min":0.308,"15min":0.308},"max":{"1min":1.236,"5min":7.5869999999999997,"15min":7.5869999999999997},"last":0.89600000000000002},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.2869999999999999,"15min":1.526},"min":{"1min":0.53800000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.423,"5min":11.494,"15min":11.494},"last":1.3819999999999999}]},{"osd":4,"last update":"Sun Sep 15 03:40:43 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":1.0449999999999999,"15min":1.0820000000000001},"min":{"1min":0.32600000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.319,"5min":10.348000000000001,"15min":10.348000000000001},"last":1.212},{"interface":"front","average":{"1min":1.083,"5min":1.0509999999999999,"15min":1.0049999999999999},"min":{"1min":0.44,"5min":0.316,"15min":0.316},"max":{"1min":1.373,"5min":7.5620000000000003,"15min":7.5620000000000003},"last":0.92900000000000005}]},{"osd":5,"last update":"Sun Sep 15 03:40:43 2024","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.86899999999999999,"15min":0.83799999999999997},"min":{"1min":0.54500000000000004,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.3400000000000001,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":1.5349999999999999},{"interface":"front","average":{"1min":1.147,"5min":1.0149999999999999,"15min":0.96699999999999997},"min":{"1min":0.56699999999999995,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.444,"5min":1.623,"15min":1.623},"last":1.5780000000000001}]},{"osd":7,"last update":"Sun Sep 15 03:40:43 2024","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.121,"15min":1.0149999999999999},"min":{"1min":0.40500000000000003,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.4279999999999999,"5min":10.923999999999999,"15min":10.923999999999999},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.1160000000000001,"15min":1.01},"min":{"1min":0.52100000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.7130000000000001,"5min":10.74,"15min":10.74},"last":1.0960000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542195,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1179920,"kb_used_data":874248,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":93191920,"statfs":{"total":96636764160,"available":95428526080,"internally_reserved":0,"allocated":895229952,"data_stored":894483388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 03:39:45 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.68300000000000005,"15min":0.65900000000000003},"min":{"1min":0.30399999999999999,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.80800000000000005,"15min":0.83999999999999997},"min":{"1min":0.30199999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.139,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":0.81299999999999994}]},{"osd":1,"last update":"Sun Sep 15 03:39:45 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.746,"15min":0.70399999999999996},"min":{"1min":0.32900000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.1259999999999999,"5min":2.302,"15min":2.302},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.71099999999999997,"15min":0.68899999999999995},"min":{"1min":0.44600000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.23,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.90000000000000002}]},{"osd":2,"last update":"Sun Sep 15 03:39:45 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.82199999999999995,"15min":0.82399999999999995},"min":{"1min":0.54500000000000004,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.0800000000000001,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85399999999999998,"15min":0.85999999999999999},"min":{"1min":0.38300000000000001,"5min":0.314,"15min":0.314},"max":{"1min":1.3839999999999999,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":1.0249999999999999}]},{"osd":3,"last update":"Sun Sep 15 03:39:45 2024","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":1.669,"15min":2.0310000000000001},"min":{"1min":0.46700000000000003,"5min":0.375,"15min":0.375},"max":{"1min":1.2629999999999999,"5min":26.664999999999999,"15min":26.664999999999999},"last":1.073},{"interface":"front","average":{"1min":0.98199999999999998,"5min":1.827,"15min":2.2189999999999999},"min":{"1min":0.49399999999999999,"5min":0.312,"15min":0.312},"max":{"1min":1.5069999999999999,"5min":26.748000000000001,"15min":26.748000000000001},"last":1.111}]},{"osd":4,"last update":"Sun Sep 15 03:39:45 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.90000000000000002,"15min":0.88},"min":{"1min":0.34999999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.4510000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":1.1830000000000001},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.90500000000000003,"15min":0.875},"min":{"1min":0.47799999999999998,"5min":0.318,"15min":0.318},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":1.222}]},{"osd":6,"last update":"Sun Sep 15 03:39:45 2024","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":0.97499999999999998,"15min":0.95999999999999996},"min":{"1min":0.55700000000000005,"5min":0.373,"15min":0.373},"max":{"1min":1.6020000000000001,"5min":1.9730000000000001,"15min":1.9730000000000001},"last":0.97699999999999998},{"interface":"front","average":{"1min":1.121,"5min":1.026,"15min":1},"min":{"1min":0.439,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.5369999999999999,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.2689999999999999}]},{"osd":7,"last update":"Sun Sep 15 03:39:45 2024","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.94299999999999995,"15min":0.90200000000000002},"min":{"1min":0.42199999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.4470000000000001,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":1.29},{"interface":"front","average":{"1min":1.252,"5min":1.1020000000000001,"15min":1.071},"min":{"1min":0.54400000000000004,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":4.3070000000000004,"5min":4.3070000000000004,"15min":4.3070000000000004},"last":1.3400000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542195,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1279268,"kb_used_data":981916,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93092572,"statfs":{"total":96636764160,"available":95326793728,"internally_reserved":0,"allocated":1005481984,"data_stored":1004653769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 03:39:41 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.64700000000000002,"15min":0.69899999999999995},"min":{"1min":0.27400000000000002,"5min":0.22,"15min":0.22},"max":{"1min":1.0509999999999999,"5min":4.7809999999999997,"15min":4.7809999999999997},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.85899999999999999,"15min":0.99099999999999999},"min":{"1min":0.314,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.98899999999999999,"5min":7.7370000000000001,"15min":7.7370000000000001},"last":0.95899999999999996}]},{"osd":1,"last update":"Sun Sep 15 03:39:41 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.746,"15min":0.627},"min":{"1min":0.27600000000000002,"5min":0.217,"15min":0.217},"max":{"1min":1.002,"5min":13.749000000000001,"15min":13.749000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.68000000000000005,"5min":1.29,"15min":0.96599999999999997},"min":{"1min":0.31900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.052,"5min":46.225999999999999,"15min":46.225999999999999},"last":1.1419999999999999}]},{"osd":2,"last update":"Sun Sep 15 03:39:41 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.71699999999999997,"15min":0.69899999999999995},"min":{"1min":0.39100000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.1379999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.72799999999999998,"15min":0.71499999999999997},"min":{"1min":0.45400000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":1.0760000000000001}]},{"osd":4,"last update":"Sun Sep 15 03:39:41 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.92600000000000005,"15min":0.94199999999999995},"min":{"1min":0.40799999999999997,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.335,"5min":2.9870000000000001,"15min":2.9870000000000001},"last":0.877},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.94299999999999995,"15min":0.96499999999999997},"min":{"1min":0.42599999999999999,"5min":0.376,"15min":0.376},"max":{"1min":1.3080000000000001,"5min":2.609,"15min":2.609},"last":1.3109999999999999}]},{"osd":5,"last update":"Sun Sep 15 03:39:41 2024","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.86599999999999999,"15min":0.84099999999999997},"min":{"1min":0.47999999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.22,"5min":1.381,"15min":1.381},"last":1.52},{"interface":"front","average":{"1min":0.86099999999999999,"5min":1.0649999999999999,"15min":1.1040000000000001},"min":{"1min":0.40100000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.242,"5min":5.0309999999999997,"15min":5.0309999999999997},"last":1.284}]},{"osd":6,"last update":"Sun Sep 15 03:39:41 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.97599999999999998,"15min":0.97299999999999998},"min":{"1min":0.54600000000000004,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.3200000000000001,"5min":1.516,"15min":1.516},"last":1.3899999999999999},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.97499999999999998,"15min":0.96299999999999997},"min":{"1min":0.59699999999999998,"5min":0.503,"15min":0.503},"max":{"1min":1.5680000000000001,"5min":2.5550000000000002,"15min":2.5550000000000002},"last":1.115}]},{"osd":7,"last update":"Sun Sep 15 03:39:41 2024","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":1.1359999999999999,"15min":1.1850000000000001},"min":{"1min":0.57799999999999996,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.494,"5min":5.5439999999999996,"15min":5.5439999999999996},"last":1.036},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.399,"15min":1.5600000000000001},"min":{"1min":0.56999999999999995,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.357,"5min":9.5850000000000009,"15min":9.5850000000000009},"last":1.466}]}]},{"osd":2,"up_from":14,"seq":60129542197,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":683092,"kb_used_data":377416,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":93688748,"statfs":{"total":96636764160,"available":95937277952,"internally_reserved":0,"allocated":386473984,"data_stored":386097365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 03:40:46 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.59699999999999998,"15min":0.61199999999999999},"min":{"1min":0.39800000000000002,"5min":0.193,"15min":0.193},"max":{"1min":0.96499999999999997,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.66700000000000004,"15min":0.65700000000000003},"min":{"1min":0.54000000000000004,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":1.228,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.59299999999999997}]},{"osd":1,"last update":"Sun Sep 15 03:40:46 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.73099999999999998,"15min":0.72599999999999998},"min":{"1min":0.47299999999999998,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.044,"5min":1.258,"15min":1.258},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.61599999999999999,"15min":0.57999999999999996},"min":{"1min":0.41999999999999998,"5min":0.222,"15min":0.222},"max":{"1min":0.92800000000000005,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.71599999999999997}]},{"osd":3,"last update":"Sun Sep 15 03:40:46 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.68300000000000005,"15min":0.66000000000000003},"min":{"1min":0.42399999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.98299999999999998,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.77800000000000002,"15min":0.73399999999999999},"min":{"1min":0.40100000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.753}]},{"osd":4,"last update":"Sun Sep 15 03:40:46 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.84099999999999997,"15min":0.77900000000000003},"min":{"1min":0.55500000000000005,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.02,"5min":0.91100000000000003,"15min":0.86599999999999999},"min":{"1min":0.53100000000000003,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.4239999999999999,"5min":1.573,"15min":1.573},"last":0.92000000000000004}]},{"osd":5,"last update":"Sun Sep 15 03:40:46 2024","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.94299999999999995,"15min":0.95399999999999996},"min":{"1min":0.68500000000000005,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.3680000000000001,"5min":4.5839999999999996,"15min":4.5839999999999996},"last":0.95299999999999996},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0549999999999999,"15min":1.113},"min":{"1min":0.50900000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.341,"5min":4.8230000000000004,"15min":4.8230000000000004},"last":1.2430000000000001}]},{"osd":6,"last update":"Sun Sep 15 03:40:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":0.94199999999999995,"15min":0.86799999999999999},"min":{"1min":0.81100000000000005,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.288,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":0.88700000000000001},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.03,"15min":0.96999999999999997},"min":{"1min":0.49299999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.425,"5min":1.9039999999999999,"15min":1.9039999999999999},"last":1.0900000000000001}]},{"osd":7,"last update":"Sun Sep 15 03:40:46 2024","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.0389999999999999,"15min":1.044},"min":{"1min":0.72599999999999998,"5min":0.379,"15min":0.379},"max":{"1min":1.4590000000000001,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":1.331},{"interface":"front","average":{"1min":1.169,"5min":1.3540000000000001,"15min":1.6080000000000001},"min":{"1min":0.74099999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.4770000000000001,"5min":10.428000000000001,"15min":10.428000000000001},"last":1.022}]}]},{"osd":4,"up_from":13,"seq":55834574901,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1618136,"kb_used_data":1320784,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":92753704,"statfs":{"total":96636764160,"available":94979792896,"internally_reserved":0,"allocated":1352482816,"data_stored":1351530757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 03:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.85799999999999998,"15min":0.93999999999999995},"min":{"1min":0.311,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.137,"5min":6.5010000000000003,"15min":6.5010000000000003},"last":1.056},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.85299999999999998,"15min":0.89300000000000002},"min":{"1min":0.34300000000000003,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.159,"5min":3.645,"15min":3.645},"last":0.80100000000000005}]},{"osd":1,"last update":"Sun Sep 15 03:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.84499999999999997,"15min":0.88900000000000001},"min":{"1min":0.29299999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.2589999999999999,"5min":3.7069999999999999,"15min":3.7069999999999999},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.81699999999999995,"15min":0.80200000000000005},"min":{"1min":0.44600000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.272,"5min":2.5779999999999998,"15min":2.5779999999999998},"last":1.093}]},{"osd":2,"last update":"Sun Sep 15 03:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.84999999999999998,"15min":0.81100000000000005},"min":{"1min":0.45000000000000001,"5min":0.311,"15min":0.311},"max":{"1min":1.212,"5min":2.766,"15min":2.766},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.84199999999999997,"15min":0.77200000000000002},"min":{"1min":0.375,"5min":0.316,"15min":0.316},"max":{"1min":1.256,"5min":3.2749999999999999,"15min":3.2749999999999999},"last":0.90600000000000003}]},{"osd":3,"last update":"Sun Sep 15 03:40:45 2024","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.0609999999999999,"15min":1.173},"min":{"1min":0.501,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.357,"5min":6.3620000000000001,"15min":6.3620000000000001},"last":0.97699999999999998},{"interface":"front","average":{"1min":1.0640000000000001,"5min":0.94899999999999995,"15min":0.87},"min":{"1min":0.58799999999999997,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.4330000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":1.1759999999999999}]},{"osd":5,"last update":"Sun Sep 15 03:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":1.0069999999999999,"15min":1.0629999999999999},"min":{"1min":0.41899999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.6890000000000001,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":1.127},{"interface":"front","average":{"1min":1.171,"5min":1.143,"15min":1.107},"min":{"1min":0.53000000000000003,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.651,"5min":4.1710000000000003,"15min":4.1710000000000003},"last":1.3200000000000001}]},{"osd":6,"last update":"Sun Sep 15 03:40:45 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.0309999999999999,"15min":1.0389999999999999},"min":{"1min":0.45800000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.6120000000000001,"5min":3.669,"15min":3.669},"last":1.1990000000000001},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.0609999999999999,"15min":0.96399999999999997},"min":{"1min":0.73799999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.581,"5min":2.5329999999999999,"15min":2.5329999999999999},"last":1.2749999999999999}]},{"osd":7,"last update":"Sun Sep 15 03:40:45 2024","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.018,"15min":0.92700000000000005},"min":{"1min":0.56599999999999995,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.498,"5min":3.238,"15min":3.238},"last":1.0269999999999999},{"interface":"front","average":{"1min":1.244,"5min":1.1299999999999999,"15min":1.109},"min":{"1min":0.68700000000000006,"5min":0.378,"15min":0.378},"max":{"1min":1.552,"5min":3.8260000000000001,"15min":3.8260000000000001},"last":1.226}]}]},{"osd":1,"up_from":13,"seq":55834574900,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":986284,"kb_used_data":676452,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":93385556,"statfs":{"total":96636764160,"available":95626809344,"internally_reserved":0,"allocated":692686848,"data_stored":692165832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 03:39:44 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.56100000000000005,"15min":0.54300000000000004},"min":{"1min":0.28699999999999998,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.94599999999999995,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.64200000000000002,"15min":0.59899999999999998},"min":{"1min":0.25700000000000001,"5min":0.187,"15min":0.187},"max":{"1min":6.6050000000000004,"5min":6.6050000000000004,"15min":6.6050000000000004},"last":0.77300000000000002}]},{"osd":2,"last update":"Sun Sep 15 03:39:44 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.621,"15min":0.625},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.171,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.746,"15min":0.73999999999999999},"min":{"1min":0.32300000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.2070000000000001,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":1.0580000000000001}]},{"osd":3,"last update":"Sun Sep 15 03:39:44 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.66800000000000004,"15min":0.63300000000000001},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.93200000000000005},{"interface":"front","average":{"1min":1.1459999999999999,"5min":0.80400000000000005,"15min":0.72499999999999998},"min":{"1min":0.59199999999999997,"5min":0.255,"15min":0.255},"max":{"1min":5.6950000000000003,"5min":5.6950000000000003,"15min":5.6950000000000003},"last":1.123}]},{"osd":4,"last update":"Sun Sep 15 03:39:44 2024","interfaces":[{"interface":"back","average":{"1min":1.498,"5min":0.99399999999999999,"15min":0.88800000000000001},"min":{"1min":0.38300000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":9.1080000000000005,"5min":9.1080000000000005,"15min":9.1080000000000005},"last":1.2729999999999999},{"interface":"front","average":{"1min":1.5980000000000001,"5min":1.0669999999999999,"15min":0.97299999999999998},"min":{"1min":0.53100000000000003,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":9.1440000000000001,"5min":9.1440000000000001,"15min":9.1440000000000001},"last":1.3740000000000001}]},{"osd":5,"last update":"Sun Sep 15 03:39:44 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.90700000000000003,"15min":0.89100000000000001},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.236,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.046,"5min":0.98899999999999999,"15min":0.96199999999999997},"min":{"1min":0.47699999999999998,"5min":0.314,"15min":0.314},"max":{"1min":1.4970000000000001,"5min":2.3580000000000001,"15min":2.3580000000000001},"last":1.4530000000000001}]},{"osd":6,"last update":"Sun Sep 15 03:39:44 2024","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":0.96899999999999997,"15min":0.92900000000000005},"min":{"1min":0.40699999999999997,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":6.4109999999999996,"5min":6.4109999999999996,"15min":6.4109999999999996},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.0289999999999999,"15min":0.999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":5.3170000000000002,"5min":5.3170000000000002,"15min":5.3170000000000002},"last":1.341}]},{"osd":7,"last update":"Sun Sep 15 03:39:44 2024","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.3340000000000001,"15min":1.45},"min":{"1min":0.46300000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":6.3730000000000002,"5min":12.821,"15min":12.821},"last":1.403},{"interface":"front","average":{"1min":1.222,"5min":1.089,"15min":1.0669999999999999},"min":{"1min":0.505,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":5.2530000000000001,"5min":5.2530000000000001,"15min":5.2530000000000001},"last":1.4890000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574901,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1362492,"kb_used_data":1056820,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":93009348,"statfs":{"total":96636764160,"available":95241572352,"internally_reserved":0,"allocated":1082183680,"data_stored":1081273737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Sep 15 03:40:41 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.59399999999999997,"15min":0.59999999999999998},"min":{"1min":0.20799999999999999,"5min":0.19,"15min":0.19},"max":{"1min":2.9279999999999999,"5min":2.9279999999999999,"15min":2.9279999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.58099999999999996,"15min":0.54400000000000004},"min":{"1min":0.41399999999999998,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.79500000000000004}]},{"osd":2,"last update":"Sun Sep 15 03:39:46 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.57499999999999996,"15min":0.54500000000000004},"min":{"1min":0.317,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.69499999999999995,"15min":0.69899999999999995},"min":{"1min":0.313,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.1839999999999999,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":0.93899999999999995}]},{"osd":3,"last update":"Sun Sep 15 03:39:46 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.70199999999999996,"15min":0.71799999999999997},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.135,"5min":1.603,"15min":1.603},"last":0.747},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.66700000000000004,"15min":0.64800000000000002},"min":{"1min":0.28499999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.1060000000000001,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.872}]},{"osd":4,"last update":"Sun Sep 15 03:40:41 2024","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.83099999999999996,"15min":0.80200000000000005},"min":{"1min":0.45900000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.1919999999999999,"5min":2.403,"15min":2.403},"last":1.036},{"interface":"front","average":{"1min":1.135,"5min":0.89700000000000002,"15min":0.83499999999999996},"min":{"1min":0.41299999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":3.1059999999999999,"5min":3.1059999999999999,"15min":3.1059999999999999},"last":1.0920000000000001}]},{"osd":5,"last update":"Sun Sep 15 03:39:46 2024","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.85399999999999998,"15min":0.81799999999999995},"min":{"1min":0.38100000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":1.002},{"interface":"front","average":{"1min":1.0169999999999999,"5min":0.91500000000000004,"15min":0.86699999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.5189999999999999,"5min":2.4319999999999999,"15min":2.4319999999999999},"last":1.3440000000000001}]},{"osd":6,"last update":"Sun Sep 15 03:39:46 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.88400000000000001,"15min":0.85799999999999998},"min":{"1min":0.53900000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.4550000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.02,"5min":1.006,"15min":1},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.425,"5min":2.5459999999999998,"15min":2.5459999999999998},"last":1.2689999999999999}]},{"osd":7,"last update":"Sun Sep 15 03:39:46 2024","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.242,"15min":1.3109999999999999},"min":{"1min":0.69699999999999995,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.4790000000000001,"5min":10.561999999999999,"15min":10.561999999999999},"last":1.179},{"interface":"front","average":{"1min":0.995,"5min":0.91300000000000003,"15min":0.88600000000000001},"min":{"1min":0.45700000000000002,"5min":0.377,"15min":0.377},"max":{"1min":1.4510000000000001,"5min":1.554,"15min":1.554},"last":1.1559999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1081413632,"data_stored":1080658160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":692445184,"data_stored":692079164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":386224128,"data_stored":386004840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1005244416,"data_stored":1004567120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1352245248,"data_stored":1351444108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":894988288,"data_stored":894396720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":357982208,"data_stored":357742880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1072160768,"data_stored":1071536856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T03:40:49.723 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-15T03:40:49.723 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T03:40:49.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.823+0000 7f5b59b22700 1 Processor -- start 2024-09-15T03:40:49.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.823+0000 7f5b59b22700 1 -- start start 2024-09-15T03:40:49.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.823+0000 7f5b59b22700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f5b5406e8b0 0x7f5b5406c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:49.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.823+0000 7f5b59b22700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f5b5413e0b0 0x7f5b5406c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:49.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.823+0000 7f5b59b22700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b5413e6b0 con 0x7f5b54063db0 2024-09-15T03:40:49.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.823+0000 7f5b59b22700 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b540a8be0 con 0x7f5b5406e8b0 2024-09-15T03:40:49.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.823+0000 7f5b59b22700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5413eb40 con 0x7f5b5413e0b0 2024-09-15T03:40:49.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.823+0000 7f5b53fff700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f5b5406e8b0 0x7f5b5406c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:49.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.823+0000 7f5b59321700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f5b5413e0b0 0x7f5b5406c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:49.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.823+0000 7f5b59321700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f5b5413e0b0 0x7f5b5406c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3301/0 says I am v2:172.21.15.110:43004/0 (socket says 172.21.15.110:43004) 2024-09-15T03:40:49.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.823+0000 7f5b59321700 1 -- 172.21.15.110:0/4063108625 learned_addr learned my addr 172.21.15.110:0/4063108625 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:40:49.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.827+0000 7f5b59321700 1 -- 172.21.15.110:0/4063108625 >> v2:172.21.15.110:3300/0 conn(0x7f5b5406e8b0 msgr2=0x7f5b5406c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:40:49.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.827+0000 7f5b59321700 1 --2- 172.21.15.110:0/4063108625 >> v2:172.21.15.110:3300/0 conn(0x7f5b5406e8b0 0x7f5b5406c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:49.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.827+0000 7f5b59321700 1 -- 172.21.15.110:0/4063108625 >> v1:172.21.15.177:6789/0 conn(0x7f5b54063db0 legacy=0x7f5b5413aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:49.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.827+0000 7f5b59321700 1 -- 172.21.15.110:0/4063108625 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b540a8e10 con 0x7f5b5413e0b0 2024-09-15T03:40:49.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.827+0000 7f5b59321700 1 --2- 172.21.15.110:0/4063108625 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f5b5413e0b0 0x7f5b5406c900 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c0099b0 tx=0x7f5b4c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0a57d909f1af5d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:49.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.827+0000 7f5b537fe700 1 -- 172.21.15.110:0/4063108625 <== mon.2 v2:172.21.15.110:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c009ea0 con 0x7f5b5413e0b0 2024-09-15T03:40:49.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.827+0000 7f5b537fe700 1 -- 172.21.15.110:0/4063108625 <== mon.2 v2:172.21.15.110:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b4c00b860 con 0x7f5b5413e0b0 2024-09-15T03:40:49.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.827+0000 7f5b537fe700 1 -- 172.21.15.110:0/4063108625 <== mon.2 v2:172.21.15.110:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c00bc00 con 0x7f5b5413e0b0 2024-09-15T03:40:49.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.827+0000 7f5b59b22700 1 -- 172.21.15.110:0/4063108625 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f5b5413e0b0 msgr2=0x7f5b5406c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:49.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.827+0000 7f5b59b22700 1 --2- 172.21.15.110:0/4063108625 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f5b5413e0b0 0x7f5b5406c900 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c0099b0 tx=0x7f5b4c009250 comp rx=0 tx=0).stop 2024-09-15T03:40:49.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.827+0000 7f5b59b22700 1 -- 172.21.15.110:0/4063108625 shutdown_connections 2024-09-15T03:40:49.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.827+0000 7f5b59b22700 1 --2- 172.21.15.110:0/4063108625 >> v2:172.21.15.110:3300/0 conn(0x7f5b5406e8b0 0x7f5b5406c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:49.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.827+0000 7f5b59b22700 1 --2- 172.21.15.110:0/4063108625 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f5b5413e0b0 0x7f5b5406c900 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:49.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.827+0000 7f5b59b22700 1 -- 172.21.15.110:0/4063108625 >> 172.21.15.110:0/4063108625 conn(0x7f5b540b1730 msgr2=0x7f5b540ad7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:40:49.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.827+0000 7f5b59b22700 1 -- 172.21.15.110:0/4063108625 shutdown_connections 2024-09-15T03:40:49.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.827+0000 7f5b59b22700 1 -- 172.21.15.110:0/4063108625 wait complete. 2024-09-15T03:40:49.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.827+0000 7f5b59b22700 1 Processor -- start 2024-09-15T03:40:49.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.827+0000 7f5b59b22700 1 -- start start 2024-09-15T03:40:49.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.831+0000 7f5b59b22700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b54063db0 0x7f5b5413d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:49.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.831+0000 7f5b59b22700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f5b5406e8b0 0x7f5b540afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:49.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.831+0000 7f5b59b22700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b5413e0b0 0x7f5b54150b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:49.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.831+0000 7f5b59b22700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b540b0990 con 0x7f5b54063db0 2024-09-15T03:40:49.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.831+0000 7f5b59b22700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b540afb70 con 0x7f5b5413e0b0 2024-09-15T03:40:49.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.831+0000 7f5b59b22700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b540b0330 con 0x7f5b5406e8b0 2024-09-15T03:40:49.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.831+0000 7f5b53fff700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f5b5406e8b0 0x7f5b540afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:49.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.831+0000 7f5b58b20700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b54063db0 0x7f5b5413d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:49.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.831+0000 7f5b53fff700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f5b5406e8b0 0x7f5b540afdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3301/0 says I am v2:172.21.15.110:43018/0 (socket says 172.21.15.110:43018) 2024-09-15T03:40:49.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.831+0000 7f5b53fff700 1 -- 172.21.15.110:0/2947343150 learned_addr learned my addr 172.21.15.110:0/2947343150 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:40:49.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.831+0000 7f5b59321700 1 --2- 172.21.15.110:0/2947343150 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b5413e0b0 0x7f5b54150b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:49.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.831+0000 7f5b53fff700 1 -- 172.21.15.110:0/2947343150 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b5413e0b0 msgr2=0x7f5b54150b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:49.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.831+0000 7f5b53fff700 1 --2- 172.21.15.110:0/2947343150 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b5413e0b0 0x7f5b54150b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:49.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.831+0000 7f5b53fff700 1 -- 172.21.15.110:0/2947343150 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b54063db0 msgr2=0x7f5b5413d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:49.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.831+0000 7f5b53fff700 1 --2- 172.21.15.110:0/2947343150 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b54063db0 0x7f5b5413d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:49.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.831+0000 7f5b53fff700 1 -- 172.21.15.110:0/2947343150 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b4c0094f0 con 0x7f5b5406e8b0 2024-09-15T03:40:49.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.831+0000 7f5b58b20700 1 --2- 172.21.15.110:0/2947343150 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b54063db0 0x7f5b5413d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T03:40:49.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.831+0000 7f5b53fff700 1 --2- 172.21.15.110:0/2947343150 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f5b5406e8b0 0x7f5b540afdf0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5b44009ef0 tx=0x7f5b4400b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:49.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.831+0000 7f5b50ff9700 1 -- 172.21.15.110:0/2947343150 <== mon.2 v2:172.21.15.110:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4400cb70 con 0x7f5b5406e8b0 2024-09-15T03:40:49.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.831+0000 7f5b50ff9700 1 -- 172.21.15.110:0/2947343150 <== mon.2 v2:172.21.15.110:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b4400f040 con 0x7f5b5406e8b0 2024-09-15T03:40:49.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.831+0000 7f5b59b22700 1 -- 172.21.15.110:0/2947343150 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b540b04a0 con 0x7f5b5406e8b0 2024-09-15T03:40:49.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.831+0000 7f5b50ff9700 1 -- 172.21.15.110:0/2947343150 <== mon.2 v2:172.21.15.110:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b44014480 con 0x7f5b5406e8b0 2024-09-15T03:40:49.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.831+0000 7f5b59b22700 1 -- 172.21.15.110:0/2947343150 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b54151280 con 0x7f5b5406e8b0 2024-09-15T03:40:49.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.831+0000 7f5b59b22700 1 -- 172.21.15.110:0/2947343150 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b540b0b00 con 0x7f5b5406e8b0 2024-09-15T03:40:49.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.835+0000 7f5b50ff9700 1 -- 172.21.15.110:0/2947343150 <== mon.2 v2:172.21.15.110:3301/0 4 ==== mgrmap(e 5) v1 ==== 85847+0+0 (secure 0 0 0) 0x7f5b440147f0 con 0x7f5b5406e8b0 2024-09-15T03:40:49.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.835+0000 7f5b50ff9700 1 --2- 172.21.15.110:0/2947343150 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f5b34067ab0 0x7f5b34069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:49.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.835+0000 7f5b58b20700 1 --2- 172.21.15.110:0/2947343150 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f5b34067ab0 0x7f5b34069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:49.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.835+0000 7f5b50ff9700 1 -- 172.21.15.110:0/2947343150 <== mon.2 v2:172.21.15.110:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5641+0+0 (secure 0 0 0) 0x7f5b44085fc0 con 0x7f5b5406e8b0 2024-09-15T03:40:49.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.835+0000 7f5b58b20700 1 --2- 172.21.15.110:0/2947343150 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f5b34067ab0 0x7f5b34069f60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c001dc0 tx=0x7f5b3c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:49.845 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:49.843+0000 7f5b50ff9700 1 -- 172.21.15.110:0/2947343150 <== mon.2 v2:172.21.15.110:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f5b440b6480 con 0x7f5b5406e8b0 2024-09-15T03:40:50.006 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.003+0000 7f5b59b22700 1 -- 172.21.15.110:0/2947343150 --> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5b540721b0 con 0x7f5b34067ab0 2024-09-15T03:40:50.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.011+0000 7f5b50ff9700 1 -- 172.21.15.110:0/2947343150 <== mgr.4104 v2:172.21.15.110:6832/22190 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+153374 (secure 0 0 0) 0x7f5b540721b0 con 0x7f5b34067ab0 2024-09-15T03:40:50.018 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-15T03:40:50.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.015+0000 7f5b59b22700 1 -- 172.21.15.110:0/2947343150 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f5b34067ab0 msgr2=0x7f5b34069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:50.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.015+0000 7f5b59b22700 1 --2- 172.21.15.110:0/2947343150 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f5b34067ab0 0x7f5b34069f60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c001dc0 tx=0x7f5b3c009040 comp rx=0 tx=0).stop 2024-09-15T03:40:50.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.015+0000 7f5b59b22700 1 -- 172.21.15.110:0/2947343150 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f5b5406e8b0 msgr2=0x7f5b540afdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:50.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.015+0000 7f5b59b22700 1 --2- 172.21.15.110:0/2947343150 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f5b5406e8b0 0x7f5b540afdf0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5b44009ef0 tx=0x7f5b4400b040 comp rx=0 tx=0).stop 2024-09-15T03:40:50.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.015+0000 7f5b59b22700 1 -- 172.21.15.110:0/2947343150 shutdown_connections 2024-09-15T03:40:50.020 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.015+0000 7f5b59b22700 1 --2- 172.21.15.110:0/2947343150 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f5b34067ab0 0x7f5b34069f60 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:50.020 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.015+0000 7f5b59b22700 1 --2- 172.21.15.110:0/2947343150 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b5413e0b0 0x7f5b54150b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:50.020 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.015+0000 7f5b59b22700 1 --2- 172.21.15.110:0/2947343150 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f5b5406e8b0 0x7f5b540afdf0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:50.020 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.015+0000 7f5b59b22700 1 --2- 172.21.15.110:0/2947343150 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b54063db0 0x7f5b5413d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:50.020 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.015+0000 7f5b59b22700 1 -- 172.21.15.110:0/2947343150 >> 172.21.15.110:0/2947343150 conn(0x7f5b540b1730 msgr2=0x7f5b5413a070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:40:50.020 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.015+0000 7f5b59b22700 1 -- 172.21.15.110:0/2947343150 shutdown_connections 2024-09-15T03:40:50.020 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.015+0000 7f5b59b22700 1 -- 172.21.15.110:0/2947343150 wait complete. 2024-09-15T03:40:50.021 INFO:teuthology.orchestra.run.smithi110.stderr:dumped all 2024-09-15T03:40:50.035 INFO:teuthology.orchestra.run.smithi110.stdout:{"pg_ready":true,"pg_map":{"version":143,"stamp":"2024-09-15T03:40:48.676511+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3570763067,"num_objects":883,"num_object_clones":0,"num_object_copies":1766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":883,"num_whiteouts":0,"num_read":2563,"num_read_kb":2563,"num_write":159105,"num_write_kb":5635359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160810,"ondisk_log_size":160810,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":9118160,"kb_used_data":6685256,"kb_used_omap":13,"kb_used_meta":2432818,"kb_avail":745856560,"statfs":{"total":773094113280,"available":763757117440,"internally_reserved":0,"allocated":6845702144,"data_stored":6840192686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2491206336},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003442"},"pg_stats":[{"pgid":"3.1f","version":"22'3025","reported_seq":3035,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T03:37:55.907201+0000","last_change":"2024-09-15T03:36:48.151935+0000","last_active":"2024-09-15T03:37:55.907201+0000","last_peered":"2024-09-15T03:37:55.907201+0000","last_clean":"2024-09-15T03:37:55.907201+0000","last_became_active":"2024-09-15T03:36:48.151625+0000","last_became_peered":"2024-09-15T03:36:48.151625+0000","last_unstale":"2024-09-15T03:37:55.907201+0000","last_undegraded":"2024-09-15T03:37:55.907201+0000","last_fullsized":"2024-09-15T03:37:55.907201+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":3025,"log_dups_size":0,"ondisk_log_size":3025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:02:31.103276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663296,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3019,"num_write_kb":136067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"23'6207","reported_seq":6522,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:40.006171+0000","last_change":"2024-09-15T03:36:48.150055+0000","last_active":"2024-09-15T03:39:40.006171+0000","last_peered":"2024-09-15T03:39:40.006171+0000","last_clean":"2024-09-15T03:39:40.006171+0000","last_became_active":"2024-09-15T03:36:48.149731+0000","last_became_peered":"2024-09-15T03:36:48.149731+0000","last_unstale":"2024-09-15T03:39:40.006171+0000","last_undegraded":"2024-09-15T03:39:40.006171+0000","last_fullsized":"2024-09-15T03:39:40.006171+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":6207,"log_dups_size":0,"ondisk_log_size":6207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:50:59.926991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113377280,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":1022,"num_read_kb":916,"num_write":6394,"num_write_kb":178051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"23'6287","reported_seq":6400,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:46.098160+0000","last_change":"2024-09-15T03:36:48.162431+0000","last_active":"2024-09-15T03:39:46.098160+0000","last_peered":"2024-09-15T03:39:46.098160+0000","last_clean":"2024-09-15T03:39:46.098160+0000","last_became_active":"2024-09-15T03:36:48.162085+0000","last_became_peered":"2024-09-15T03:36:48.162085+0000","last_unstale":"2024-09-15T03:39:46.098160+0000","last_undegraded":"2024-09-15T03:39:46.098160+0000","last_fullsized":"2024-09-15T03:39:46.098160+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":6287,"log_dups_size":0,"ondisk_log_size":6287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:51:54.108247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121898016,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":109,"num_read_kb":107,"num_write":6275,"num_write_kb":177699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"22'4656","reported_seq":4674,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T03:37:56.481330+0000","last_change":"2024-09-15T03:36:48.159640+0000","last_active":"2024-09-15T03:37:56.481330+0000","last_peered":"2024-09-15T03:37:56.481330+0000","last_clean":"2024-09-15T03:37:56.481330+0000","last_became_active":"2024-09-15T03:36:48.159272+0000","last_became_peered":"2024-09-15T03:36:48.159272+0000","last_unstale":"2024-09-15T03:37:56.481330+0000","last_undegraded":"2024-09-15T03:37:56.481330+0000","last_fullsized":"2024-09-15T03:37:56.481330+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":4656,"log_dups_size":0,"ondisk_log_size":4656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:17:25.869833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88866816,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":4665,"num_write_kb":169740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"23'4796","reported_seq":4806,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:32.771695+0000","last_change":"2024-09-15T03:36:48.150332+0000","last_active":"2024-09-15T03:39:32.771695+0000","last_peered":"2024-09-15T03:39:32.771695+0000","last_clean":"2024-09-15T03:39:32.771695+0000","last_became_active":"2024-09-15T03:36:48.150057+0000","last_became_peered":"2024-09-15T03:36:48.150057+0000","last_unstale":"2024-09-15T03:39:32.771695+0000","last_undegraded":"2024-09-15T03:39:32.771695+0000","last_fullsized":"2024-09-15T03:39:32.771695+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":4796,"log_dups_size":0,"ondisk_log_size":4796,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:15:39.747133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117440512,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4750,"num_write_kb":185221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"23'4713","reported_seq":4728,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:38.053359+0000","last_change":"2024-09-15T03:36:48.144892+0000","last_active":"2024-09-15T03:39:38.053359+0000","last_peered":"2024-09-15T03:39:38.053359+0000","last_clean":"2024-09-15T03:39:38.053359+0000","last_became_active":"2024-09-15T03:36:48.144097+0000","last_became_peered":"2024-09-15T03:36:48.144097+0000","last_unstale":"2024-09-15T03:39:38.053359+0000","last_undegraded":"2024-09-15T03:39:38.053359+0000","last_fullsized":"2024-09-15T03:39:38.053359+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":4713,"log_dups_size":0,"ondisk_log_size":4713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:16:01.325609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68681728,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":22,"num_read_kb":16,"num_write":4699,"num_write_kb":143618,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"23'4491","reported_seq":4502,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:50.201809+0000","last_change":"2024-09-15T03:36:48.142399+0000","last_active":"2024-09-15T03:39:50.201809+0000","last_peered":"2024-09-15T03:39:50.201809+0000","last_clean":"2024-09-15T03:39:50.201809+0000","last_became_active":"2024-09-15T03:36:48.142085+0000","last_became_peered":"2024-09-15T03:36:48.142085+0000","last_unstale":"2024-09-15T03:39:50.201809+0000","last_undegraded":"2024-09-15T03:39:50.201809+0000","last_fullsized":"2024-09-15T03:39:50.201809+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":4491,"log_dups_size":0,"ondisk_log_size":4491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T13:00:10.524261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":111872344,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":4448,"num_write_kb":183583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"23'7205","reported_seq":7219,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:50.436933+0000","last_change":"2024-09-15T03:36:48.138540+0000","last_active":"2024-09-15T03:39:50.436933+0000","last_peered":"2024-09-15T03:39:50.436933+0000","last_clean":"2024-09-15T03:39:50.436933+0000","last_became_active":"2024-09-15T03:36:48.138049+0000","last_became_peered":"2024-09-15T03:36:48.138049+0000","last_unstale":"2024-09-15T03:39:50.436933+0000","last_undegraded":"2024-09-15T03:39:50.436933+0000","last_fullsized":"2024-09-15T03:39:50.436933+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":7205,"log_dups_size":0,"ondisk_log_size":7205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T12:37:15.684685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113246720,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":15,"num_read_kb":11,"num_write":7218,"num_write_kb":193736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"23'7862","reported_seq":7879,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:54.842187+0000","last_change":"2024-09-15T03:36:48.150100+0000","last_active":"2024-09-15T03:39:54.842187+0000","last_peered":"2024-09-15T03:39:54.842187+0000","last_clean":"2024-09-15T03:39:54.842187+0000","last_became_active":"2024-09-15T03:36:48.149792+0000","last_became_peered":"2024-09-15T03:36:48.149792+0000","last_unstale":"2024-09-15T03:39:54.842187+0000","last_undegraded":"2024-09-15T03:39:54.842187+0000","last_fullsized":"2024-09-15T03:39:54.842187+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":7862,"log_dups_size":0,"ondisk_log_size":7862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:42:07.633298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":146800640,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":7874,"num_write_kb":194640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"23'2408","reported_seq":2421,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:32.679416+0000","last_change":"2024-09-15T03:36:48.155132+0000","last_active":"2024-09-15T03:39:32.679416+0000","last_peered":"2024-09-15T03:39:32.679416+0000","last_clean":"2024-09-15T03:39:32.679416+0000","last_became_active":"2024-09-15T03:36:48.154800+0000","last_became_peered":"2024-09-15T03:36:48.154800+0000","last_unstale":"2024-09-15T03:39:32.679416+0000","last_undegraded":"2024-09-15T03:39:32.679416+0000","last_fullsized":"2024-09-15T03:39:32.679416+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":2408,"log_dups_size":0,"ondisk_log_size":2408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:54:21.705564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":2388,"num_write_kb":127362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"23'6301","reported_seq":6317,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:48.732682+0000","last_change":"2024-09-15T03:36:48.152499+0000","last_active":"2024-09-15T03:39:48.732682+0000","last_peered":"2024-09-15T03:39:48.732682+0000","last_clean":"2024-09-15T03:39:48.732682+0000","last_became_active":"2024-09-15T03:36:48.152191+0000","last_became_peered":"2024-09-15T03:36:48.152191+0000","last_unstale":"2024-09-15T03:39:48.732682+0000","last_undegraded":"2024-09-15T03:39:48.732682+0000","last_fullsized":"2024-09-15T03:39:48.732682+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":6301,"log_dups_size":0,"ondisk_log_size":6301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T12:12:56.810421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":124518928,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":19,"num_read_kb":14,"num_write":6316,"num_write_kb":168780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"23'5385","reported_seq":5398,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:42.445288+0000","last_change":"2024-09-15T03:36:48.154566+0000","last_active":"2024-09-15T03:39:42.445288+0000","last_peered":"2024-09-15T03:39:42.445288+0000","last_clean":"2024-09-15T03:39:42.445288+0000","last_became_active":"2024-09-15T03:36:48.153926+0000","last_became_peered":"2024-09-15T03:36:48.153926+0000","last_unstale":"2024-09-15T03:39:42.445288+0000","last_undegraded":"2024-09-15T03:39:42.445288+0000","last_fullsized":"2024-09-15T03:39:42.445288+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":5385,"log_dups_size":0,"ondisk_log_size":5385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:19:06.740804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109121016,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":5342,"num_write_kb":144586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:32.448890+0000","last_change":"2024-09-15T03:36:42.814645+0000","last_active":"2024-09-15T03:39:32.448890+0000","last_peered":"2024-09-15T03:39:32.448890+0000","last_clean":"2024-09-15T03:39:32.448890+0000","last_became_active":"2024-09-15T03:36:40.672828+0000","last_became_peered":"2024-09-15T03:36:40.672828+0000","last_unstale":"2024-09-15T03:39:32.448890+0000","last_undegraded":"2024-09-15T03:39:32.448890+0000","last_fullsized":"2024-09-15T03:39:32.448890+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_clean_scrub_stamp":"2024-09-15T03:36:39.639431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T13:40:53.039569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064069099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'5981","reported_seq":6014,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:54.327858+0000","last_change":"2024-09-15T03:36:48.143463+0000","last_active":"2024-09-15T03:39:54.327858+0000","last_peered":"2024-09-15T03:39:54.327858+0000","last_clean":"2024-09-15T03:39:54.327858+0000","last_became_active":"2024-09-15T03:36:48.143177+0000","last_became_peered":"2024-09-15T03:36:48.143177+0000","last_unstale":"2024-09-15T03:39:54.327858+0000","last_undegraded":"2024-09-15T03:39:54.327858+0000","last_fullsized":"2024-09-15T03:39:54.327858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":5981,"log_dups_size":0,"ondisk_log_size":5981,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:20:24.926833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":134639616,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":92,"num_read_kb":80,"num_write":5989,"num_write_kb":192351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T03:36:51.577473+0000","last_change":"2024-09-15T03:36:42.687267+0000","last_active":"2024-09-15T03:36:51.577473+0000","last_peered":"2024-09-15T03:36:51.577473+0000","last_clean":"2024-09-15T03:36:51.577473+0000","last_became_active":"2024-09-15T03:36:40.671677+0000","last_became_peered":"2024-09-15T03:36:40.671677+0000","last_unstale":"2024-09-15T03:36:51.577473+0000","last_undegraded":"2024-09-15T03:36:51.577473+0000","last_fullsized":"2024-09-15T03:36:51.577473+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_clean_scrub_stamp":"2024-09-15T03:36:39.639431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:55:15.493964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000861089,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"23'5400","reported_seq":5410,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:32.770837+0000","last_change":"2024-09-15T03:36:48.152124+0000","last_active":"2024-09-15T03:39:32.770837+0000","last_peered":"2024-09-15T03:39:32.770837+0000","last_clean":"2024-09-15T03:39:32.770837+0000","last_became_active":"2024-09-15T03:36:48.151844+0000","last_became_peered":"2024-09-15T03:36:48.151844+0000","last_unstale":"2024-09-15T03:39:32.770837+0000","last_undegraded":"2024-09-15T03:39:32.770837+0000","last_fullsized":"2024-09-15T03:39:32.770837+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":5400,"log_dups_size":0,"ondisk_log_size":5400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:17:16.444530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":115998720,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5352,"num_write_kb":174984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T03:36:51.579975+0000","last_change":"2024-09-15T03:36:42.854270+0000","last_active":"2024-09-15T03:36:51.579975+0000","last_peered":"2024-09-15T03:36:51.579975+0000","last_clean":"2024-09-15T03:36:51.579975+0000","last_became_active":"2024-09-15T03:36:41.079216+0000","last_became_peered":"2024-09-15T03:36:41.079216+0000","last_unstale":"2024-09-15T03:36:51.579975+0000","last_undegraded":"2024-09-15T03:36:51.579975+0000","last_fullsized":"2024-09-15T03:36:51.579975+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_clean_scrub_stamp":"2024-09-15T03:36:39.639431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:19:37.610607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062568200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"23'7270","reported_seq":7291,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:53.195871+0000","last_change":"2024-09-15T03:36:48.216882+0000","last_active":"2024-09-15T03:39:53.195871+0000","last_peered":"2024-09-15T03:39:53.195871+0000","last_clean":"2024-09-15T03:39:53.195871+0000","last_became_active":"2024-09-15T03:36:48.216337+0000","last_became_peered":"2024-09-15T03:36:48.216337+0000","last_unstale":"2024-09-15T03:39:53.195871+0000","last_undegraded":"2024-09-15T03:39:53.195871+0000","last_fullsized":"2024-09-15T03:39:53.195871+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":7270,"log_dups_size":0,"ondisk_log_size":7270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:27:25.000492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":146931712,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":45,"num_read_kb":34,"num_write":7284,"num_write_kb":246402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T03:36:51.577450+0000","last_change":"2024-09-15T03:36:42.687270+0000","last_active":"2024-09-15T03:36:51.577450+0000","last_peered":"2024-09-15T03:36:51.577450+0000","last_clean":"2024-09-15T03:36:51.577450+0000","last_became_active":"2024-09-15T03:36:41.077266+0000","last_became_peered":"2024-09-15T03:36:41.077266+0000","last_unstale":"2024-09-15T03:36:51.577450+0000","last_undegraded":"2024-09-15T03:36:51.577450+0000","last_fullsized":"2024-09-15T03:36:51.577450+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_clean_scrub_stamp":"2024-09-15T03:36:39.639431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:54:58.435053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067514400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"22'3282","reported_seq":3292,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T03:37:53.949949+0000","last_change":"2024-09-15T03:36:48.151841+0000","last_active":"2024-09-15T03:37:53.949949+0000","last_peered":"2024-09-15T03:37:53.949949+0000","last_clean":"2024-09-15T03:37:53.949949+0000","last_became_active":"2024-09-15T03:36:48.151520+0000","last_became_peered":"2024-09-15T03:36:48.151520+0000","last_unstale":"2024-09-15T03:37:53.949949+0000","last_undegraded":"2024-09-15T03:37:53.949949+0000","last_fullsized":"2024-09-15T03:37:53.949949+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":3282,"log_dups_size":0,"ondisk_log_size":3282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:24:47.104486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3278,"num_write_kb":144390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"22'3449","reported_seq":3473,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T03:37:58.123758+0000","last_change":"2024-09-15T03:36:48.146454+0000","last_active":"2024-09-15T03:37:58.123758+0000","last_peered":"2024-09-15T03:37:58.123758+0000","last_clean":"2024-09-15T03:37:58.123758+0000","last_became_active":"2024-09-15T03:36:48.145845+0000","last_became_peered":"2024-09-15T03:36:48.145845+0000","last_unstale":"2024-09-15T03:37:58.123758+0000","last_undegraded":"2024-09-15T03:37:58.123758+0000","last_fullsized":"2024-09-15T03:37:58.123758+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":3449,"log_dups_size":0,"ondisk_log_size":3449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:46:57.211180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113246208,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":50,"num_read_kb":41,"num_write":3479,"num_write_kb":144783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T03:36:50.383528+0000","last_change":"2024-09-15T03:36:42.695885+0000","last_active":"2024-09-15T03:36:50.383528+0000","last_peered":"2024-09-15T03:36:50.383528+0000","last_clean":"2024-09-15T03:36:50.383528+0000","last_became_active":"2024-09-15T03:36:40.673743+0000","last_became_peered":"2024-09-15T03:36:40.673743+0000","last_unstale":"2024-09-15T03:36:50.383528+0000","last_undegraded":"2024-09-15T03:36:50.383528+0000","last_fullsized":"2024-09-15T03:36:50.383528+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_clean_scrub_stamp":"2024-09-15T03:36:39.639431+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T13:57:31.462981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002159518,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"23'4710","reported_seq":4734,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:32.771799+0000","last_change":"2024-09-15T03:36:48.216887+0000","last_active":"2024-09-15T03:39:32.771799+0000","last_peered":"2024-09-15T03:39:32.771799+0000","last_clean":"2024-09-15T03:39:32.771799+0000","last_became_active":"2024-09-15T03:36:48.216269+0000","last_became_peered":"2024-09-15T03:36:48.216269+0000","last_unstale":"2024-09-15T03:39:32.771799+0000","last_undegraded":"2024-09-15T03:39:32.771799+0000","last_fullsized":"2024-09-15T03:39:32.771799+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":4710,"log_dups_size":0,"ondisk_log_size":4710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T12:10:11.804094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":142877200,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":51,"num_read_kb":164,"num_write":4698,"num_write_kb":182422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:32.402704+0000","last_change":"2024-09-15T03:36:42.801569+0000","last_active":"2024-09-15T03:39:32.402704+0000","last_peered":"2024-09-15T03:39:32.402704+0000","last_clean":"2024-09-15T03:39:32.402704+0000","last_became_active":"2024-09-15T03:36:40.669392+0000","last_became_peered":"2024-09-15T03:36:40.669392+0000","last_unstale":"2024-09-15T03:39:32.402704+0000","last_undegraded":"2024-09-15T03:39:32.402704+0000","last_fullsized":"2024-09-15T03:39:32.402704+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_clean_scrub_stamp":"2024-09-15T03:36:39.639431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:00:33.102146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059629799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"23'3227","reported_seq":3245,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:32.683131+0000","last_change":"2024-09-15T03:36:48.144585+0000","last_active":"2024-09-15T03:39:32.683131+0000","last_peered":"2024-09-15T03:39:32.683131+0000","last_clean":"2024-09-15T03:39:32.683131+0000","last_became_active":"2024-09-15T03:36:48.144313+0000","last_became_peered":"2024-09-15T03:36:48.144313+0000","last_unstale":"2024-09-15T03:39:32.683131+0000","last_undegraded":"2024-09-15T03:39:32.683131+0000","last_fullsized":"2024-09-15T03:39:32.683131+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":3227,"log_dups_size":0,"ondisk_log_size":3227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T15:27:39.715172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92405760,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":33,"num_read_kb":87,"num_write":3327,"num_write_kb":128260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T03:36:51.579392+0000","last_change":"2024-09-15T03:36:42.854222+0000","last_active":"2024-09-15T03:36:51.579392+0000","last_peered":"2024-09-15T03:36:51.579392+0000","last_clean":"2024-09-15T03:36:51.579392+0000","last_became_active":"2024-09-15T03:36:40.672416+0000","last_became_peered":"2024-09-15T03:36:40.672416+0000","last_unstale":"2024-09-15T03:36:51.579392+0000","last_undegraded":"2024-09-15T03:36:51.579392+0000","last_fullsized":"2024-09-15T03:36:51.579392+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_clean_scrub_stamp":"2024-09-15T03:36:39.639431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T15:08:21.420921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065187600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"23'3107","reported_seq":3117,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:46.717682+0000","last_change":"2024-09-15T03:36:48.151148+0000","last_active":"2024-09-15T03:39:46.717682+0000","last_peered":"2024-09-15T03:39:46.717682+0000","last_clean":"2024-09-15T03:39:46.717682+0000","last_became_active":"2024-09-15T03:36:48.150693+0000","last_became_peered":"2024-09-15T03:36:48.150693+0000","last_unstale":"2024-09-15T03:39:46.717682+0000","last_undegraded":"2024-09-15T03:39:46.717682+0000","last_fullsized":"2024-09-15T03:39:46.717682+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":3107,"log_dups_size":0,"ondisk_log_size":3107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:27:52.876606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67108864,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3105,"num_write_kb":103402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T03:36:50.383548+0000","last_change":"2024-09-15T03:36:42.694486+0000","last_active":"2024-09-15T03:36:50.383548+0000","last_peered":"2024-09-15T03:36:50.383548+0000","last_clean":"2024-09-15T03:36:50.383548+0000","last_became_active":"2024-09-15T03:36:40.674442+0000","last_became_peered":"2024-09-15T03:36:40.674442+0000","last_unstale":"2024-09-15T03:36:50.383548+0000","last_undegraded":"2024-09-15T03:36:50.383548+0000","last_fullsized":"2024-09-15T03:36:50.383548+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_clean_scrub_stamp":"2024-09-15T03:36:39.639431+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:25:57.474430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067001500000000004,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":106,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T03:36:51.579500+0000","last_change":"2024-09-15T03:36:38.815652+0000","last_active":"2024-09-15T03:36:51.579500+0000","last_peered":"2024-09-15T03:36:51.579500+0000","last_clean":"2024-09-15T03:36:51.579500+0000","last_became_active":"2024-09-15T03:36:38.813490+0000","last_became_peered":"2024-09-15T03:36:38.813490+0000","last_unstale":"2024-09-15T03:36:51.579500+0000","last_undegraded":"2024-09-15T03:36:51.579500+0000","last_fullsized":"2024-09-15T03:36:51.579500+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:37.618040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:37.618040+0000","last_clean_scrub_stamp":"2024-09-15T03:36:37.618040+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:02:12.095255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"23'5187","reported_seq":5201,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:32.773823+0000","last_change":"2024-09-15T03:36:48.142652+0000","last_active":"2024-09-15T03:39:32.773823+0000","last_peered":"2024-09-15T03:39:32.773823+0000","last_clean":"2024-09-15T03:39:32.773823+0000","last_became_active":"2024-09-15T03:36:48.142313+0000","last_became_peered":"2024-09-15T03:36:48.142313+0000","last_unstale":"2024-09-15T03:39:32.773823+0000","last_undegraded":"2024-09-15T03:39:32.773823+0000","last_fullsized":"2024-09-15T03:39:32.773823+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":5187,"log_dups_size":0,"ondisk_log_size":5187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T13:50:10.152068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":116760576,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":12,"num_read_kb":135,"num_write":5130,"num_write_kb":177200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"23'7935","reported_seq":8051,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:49.720540+0000","last_change":"2024-09-15T03:36:48.218591+0000","last_active":"2024-09-15T03:39:49.720540+0000","last_peered":"2024-09-15T03:39:49.720540+0000","last_clean":"2024-09-15T03:39:49.720540+0000","last_became_active":"2024-09-15T03:36:48.218083+0000","last_became_peered":"2024-09-15T03:36:48.218083+0000","last_unstale":"2024-09-15T03:39:49.720540+0000","last_undegraded":"2024-09-15T03:39:49.720540+0000","last_fullsized":"2024-09-15T03:39:49.720540+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":7935,"log_dups_size":0,"ondisk_log_size":7935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:15:32.571275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663808,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":356,"num_read_kb":379,"num_write":5975,"num_write_kb":152245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"24'5837","reported_seq":5868,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:48.324243+0000","last_change":"2024-09-15T03:36:48.155747+0000","last_active":"2024-09-15T03:40:48.324243+0000","last_peered":"2024-09-15T03:40:48.324243+0000","last_clean":"2024-09-15T03:40:48.324243+0000","last_became_active":"2024-09-15T03:36:48.155263+0000","last_became_peered":"2024-09-15T03:36:48.155263+0000","last_unstale":"2024-09-15T03:40:48.324243+0000","last_undegraded":"2024-09-15T03:40:48.324243+0000","last_fullsized":"2024-09-15T03:40:48.324243+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":5837,"log_dups_size":0,"ondisk_log_size":5837,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:40:42.479597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":137103712,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":68,"num_read_kb":57,"num_write":5841,"num_write_kb":205281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"22'3150","reported_seq":3160,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T03:37:55.878205+0000","last_change":"2024-09-15T03:36:48.138000+0000","last_active":"2024-09-15T03:37:55.878205+0000","last_peered":"2024-09-15T03:37:55.878205+0000","last_clean":"2024-09-15T03:37:55.878205+0000","last_became_active":"2024-09-15T03:36:48.137589+0000","last_became_peered":"2024-09-15T03:36:48.137589+0000","last_unstale":"2024-09-15T03:37:55.878205+0000","last_undegraded":"2024-09-15T03:37:55.878205+0000","last_fullsized":"2024-09-15T03:37:55.878205+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":3150,"log_dups_size":0,"ondisk_log_size":3150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:02:31.208417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3149,"num_write_kb":152324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"23'4730","reported_seq":4757,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:32.766647+0000","last_change":"2024-09-15T03:36:48.151666+0000","last_active":"2024-09-15T03:39:32.766647+0000","last_peered":"2024-09-15T03:39:32.766647+0000","last_clean":"2024-09-15T03:39:32.766647+0000","last_became_active":"2024-09-15T03:36:48.151365+0000","last_became_peered":"2024-09-15T03:36:48.151365+0000","last_unstale":"2024-09-15T03:39:32.766647+0000","last_undegraded":"2024-09-15T03:39:32.766647+0000","last_fullsized":"2024-09-15T03:39:32.766647+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":4730,"log_dups_size":0,"ondisk_log_size":4730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T15:05:31.496486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121635328,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":72,"num_read_kb":60,"num_write":4709,"num_write_kb":215880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"23'5517","reported_seq":5531,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:32.678901+0000","last_change":"2024-09-15T03:36:48.155332+0000","last_active":"2024-09-15T03:39:32.678901+0000","last_peered":"2024-09-15T03:39:32.678901+0000","last_clean":"2024-09-15T03:39:32.678901+0000","last_became_active":"2024-09-15T03:36:48.154955+0000","last_became_peered":"2024-09-15T03:36:48.154955+0000","last_unstale":"2024-09-15T03:39:32.678901+0000","last_undegraded":"2024-09-15T03:39:32.678901+0000","last_fullsized":"2024-09-15T03:39:32.678901+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":5517,"log_dups_size":0,"ondisk_log_size":5517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:17:29.191405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117440512,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":5499,"num_write_kb":248958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"23'4086","reported_seq":4166,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:32.831744+0000","last_change":"2024-09-15T03:36:48.162708+0000","last_active":"2024-09-15T03:39:32.831744+0000","last_peered":"2024-09-15T03:39:32.831744+0000","last_clean":"2024-09-15T03:39:32.831744+0000","last_became_active":"2024-09-15T03:36:48.162401+0000","last_became_peered":"2024-09-15T03:36:48.162401+0000","last_unstale":"2024-09-15T03:39:32.831744+0000","last_undegraded":"2024-09-15T03:39:32.831744+0000","last_fullsized":"2024-09-15T03:39:32.831744+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":4086,"log_dups_size":0,"ondisk_log_size":4086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:15:25.156037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104866320,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":269,"num_read_kb":201,"num_write":4343,"num_write_kb":177773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"23'3614","reported_seq":3643,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:32.772767+0000","last_change":"2024-09-15T03:36:48.162180+0000","last_active":"2024-09-15T03:39:32.772767+0000","last_peered":"2024-09-15T03:39:32.772767+0000","last_clean":"2024-09-15T03:39:32.772767+0000","last_became_active":"2024-09-15T03:36:48.161897+0000","last_became_peered":"2024-09-15T03:36:48.161897+0000","last_unstale":"2024-09-15T03:39:32.772767+0000","last_undegraded":"2024-09-15T03:39:32.772767+0000","last_fullsized":"2024-09-15T03:39:32.772767+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":3614,"log_dups_size":0,"ondisk_log_size":3614,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:44:27.564730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":130023424,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":75,"num_read_kb":60,"num_write":3605,"num_write_kb":185220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"24'4055","reported_seq":4066,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:48.332171+0000","last_change":"2024-09-15T03:36:48.149972+0000","last_active":"2024-09-15T03:40:48.332171+0000","last_peered":"2024-09-15T03:40:48.332171+0000","last_clean":"2024-09-15T03:40:48.332171+0000","last_became_active":"2024-09-15T03:36:48.149651+0000","last_became_peered":"2024-09-15T03:36:48.149651+0000","last_unstale":"2024-09-15T03:40:48.332171+0000","last_undegraded":"2024-09-15T03:40:48.332171+0000","last_fullsized":"2024-09-15T03:40:48.332171+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":4055,"log_dups_size":0,"ondisk_log_size":4055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T15:32:38.282915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":111869952,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4006,"num_write_kb":198851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"22'2797","reported_seq":2823,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T03:37:53.771805+0000","last_change":"2024-09-15T03:36:48.159682+0000","last_active":"2024-09-15T03:37:53.771805+0000","last_peered":"2024-09-15T03:37:53.771805+0000","last_clean":"2024-09-15T03:37:53.771805+0000","last_became_active":"2024-09-15T03:36:48.159380+0000","last_became_peered":"2024-09-15T03:36:48.159380+0000","last_unstale":"2024-09-15T03:37:53.771805+0000","last_undegraded":"2024-09-15T03:37:53.771805+0000","last_fullsized":"2024-09-15T03:37:53.771805+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":2797,"log_dups_size":0,"ondisk_log_size":2797,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T13:43:53.575032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":50,"num_read_kb":41,"num_write":2819,"num_write_kb":144138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"23'5712","reported_seq":5727,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:32.775461+0000","last_change":"2024-09-15T03:36:48.151085+0000","last_active":"2024-09-15T03:39:32.775461+0000","last_peered":"2024-09-15T03:39:32.775461+0000","last_clean":"2024-09-15T03:39:32.775461+0000","last_became_active":"2024-09-15T03:36:48.150784+0000","last_became_peered":"2024-09-15T03:36:48.150784+0000","last_unstale":"2024-09-15T03:39:32.775461+0000","last_undegraded":"2024-09-15T03:39:32.775461+0000","last_fullsized":"2024-09-15T03:39:32.775461+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":5712,"log_dups_size":0,"ondisk_log_size":5712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:11:11.620798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":138543104,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":22,"num_read_kb":16,"num_write":5677,"num_write_kb":255624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"23'8353","reported_seq":8372,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:46.094057+0000","last_change":"2024-09-15T03:36:48.159758+0000","last_active":"2024-09-15T03:39:46.094057+0000","last_peered":"2024-09-15T03:39:46.094057+0000","last_clean":"2024-09-15T03:39:46.094057+0000","last_became_active":"2024-09-15T03:36:48.159470+0000","last_became_peered":"2024-09-15T03:36:48.159470+0000","last_unstale":"2024-09-15T03:39:46.094057+0000","last_undegraded":"2024-09-15T03:39:46.094057+0000","last_fullsized":"2024-09-15T03:39:46.094057+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":8353,"log_dups_size":0,"ondisk_log_size":8353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:06:20.271144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":126026248,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":36,"num_read_kb":28,"num_write":8349,"num_write_kb":200158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":3570172680,"num_objects":879,"num_object_clones":0,"num_object_copies":1758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":879,"num_whiteouts":0,"num_read":2497,"num_read_kb":2508,"num_write":159002,"num_write_kb":5633729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":6842703872,"data_stored":6838429848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160735,"ondisk_log_size":160735,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542196,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1357616,"kb_used_data":1047784,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":93014224,"statfs":{"total":96636764160,"available":95246565376,"internally_reserved":0,"allocated":1072930816,"data_stored":1072152433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 03:39:47 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":2.0049999999999999,"15min":2.5830000000000002},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.1799999999999999,"5min":35.189,"15min":35.189},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.749,"15min":0.74199999999999999},"min":{"1min":0.39900000000000002,"5min":0.245,"15min":0.245},"max":{"1min":1.173,"5min":1.427,"15min":1.427},"last":0.76500000000000001}]},{"osd":1,"last update":"Sun Sep 15 03:39:47 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":2.0150000000000001,"15min":2.544},"min":{"1min":0.315,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.54,"5min":35.223999999999997,"15min":35.223999999999997},"last":1.1779999999999999},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78700000000000003,"15min":0.76700000000000002},"min":{"1min":0.38300000000000001,"5min":0.308,"15min":0.308},"max":{"1min":1.1930000000000001,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.84799999999999998}]},{"osd":2,"last update":"Sun Sep 15 03:39:47 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.73199999999999998,"15min":0.72899999999999998},"min":{"1min":0.374,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.2509999999999999,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":1.0409999999999999},{"interface":"front","average":{"1min":0.81799999999999995,"5min":2.105,"15min":2.641},"min":{"1min":0.40899999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.3879999999999999,"5min":35.283000000000001,"15min":35.283000000000001},"last":1.0149999999999999}]},{"osd":3,"last update":"Sun Sep 15 03:39:47 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.78400000000000003,"15min":0.74099999999999999},"min":{"1min":0.45500000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.266,"5min":1.71,"15min":1.71},"last":1.117},{"interface":"front","average":{"1min":0.92000000000000004,"5min":2.1749999999999998,"15min":2.6930000000000001},"min":{"1min":0.50800000000000001,"5min":0.378,"15min":0.378},"max":{"1min":1.3580000000000001,"5min":35.255000000000003,"15min":35.255000000000003},"last":0.97999999999999998}]},{"osd":4,"last update":"Sun Sep 15 03:39:47 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":1.048,"15min":0.80800000000000005},"min":{"1min":0.33900000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.3380000000000001,"5min":25.52,"15min":25.52},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.81699999999999995,"15min":0.77300000000000002},"min":{"1min":0.32200000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.339,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":1.1499999999999999}]},{"osd":5,"last update":"Sun Sep 15 03:39:47 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":2.238,"15min":2.778},"min":{"1min":0.41599999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.329,"5min":35.070999999999998,"15min":35.070999999999998},"last":1.2290000000000001},{"interface":"front","average":{"1min":0.96999999999999997,"5min":1.0820000000000001,"15min":1.101},"min":{"1min":0.43099999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.4430000000000001,"5min":5.2999999999999998,"15min":5.2999999999999998},"last":1.2}]},{"osd":6,"last update":"Sun Sep 15 03:39:47 2024","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.94399999999999995,"15min":0.878},"min":{"1min":0.42099999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.355,"5min":2.5449999999999999,"15min":2.5449999999999999},"last":1.085},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.90700000000000003,"15min":0.84699999999999998},"min":{"1min":0.40799999999999997,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.484,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":1.452}]}]},{"osd":6,"up_from":14,"seq":60129542196,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":651352,"kb_used_data":349836,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93720488,"statfs":{"total":96636764160,"available":95969779712,"internally_reserved":0,"allocated":358232064,"data_stored":357835405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 03:40:43 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":1.0049999999999999,"15min":1.155},"min":{"1min":0.253,"5min":0.25,"15min":0.25},"max":{"1min":1.208,"5min":7.3170000000000002,"15min":7.3170000000000002},"last":1.3069999999999999},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.82399999999999995,"15min":0.77300000000000002},"min":{"1min":0.47799999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.212,"5min":7.3849999999999998,"15min":7.3849999999999998},"last":1.157}]},{"osd":1,"last update":"Sun Sep 15 03:40:43 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.72899999999999998,"15min":0.69099999999999995},"min":{"1min":0.30299999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.157,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.89600000000000002,"15min":0.91800000000000004},"min":{"1min":0.45800000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.304,"5min":7.3559999999999999,"15min":7.3559999999999999},"last":1.1100000000000001}]},{"osd":2,"last update":"Sun Sep 15 03:40:43 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.92800000000000005,"15min":0.93500000000000005},"min":{"1min":0.436,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.286,"5min":7.4960000000000004,"15min":7.4960000000000004},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.84699999999999998,"15min":0.88600000000000001},"min":{"1min":0.49199999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.1499999999999999,"5min":1.575,"15min":1.575},"last":1.474}]},{"osd":3,"last update":"Sun Sep 15 03:40:43 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":1.0569999999999999,"15min":1.1499999999999999},"min":{"1min":0.42399999999999999,"5min":0.308,"15min":0.308},"max":{"1min":1.236,"5min":7.5869999999999997,"15min":7.5869999999999997},"last":0.89600000000000002},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.2869999999999999,"15min":1.526},"min":{"1min":0.53800000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.423,"5min":11.494,"15min":11.494},"last":1.3819999999999999}]},{"osd":4,"last update":"Sun Sep 15 03:40:43 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":1.0449999999999999,"15min":1.0820000000000001},"min":{"1min":0.32600000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.319,"5min":10.348000000000001,"15min":10.348000000000001},"last":1.212},{"interface":"front","average":{"1min":1.083,"5min":1.0509999999999999,"15min":1.0049999999999999},"min":{"1min":0.44,"5min":0.316,"15min":0.316},"max":{"1min":1.373,"5min":7.5620000000000003,"15min":7.5620000000000003},"last":0.92900000000000005}]},{"osd":5,"last update":"Sun Sep 15 03:40:43 2024","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.86899999999999999,"15min":0.83799999999999997},"min":{"1min":0.54500000000000004,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.3400000000000001,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":1.5349999999999999},{"interface":"front","average":{"1min":1.147,"5min":1.0149999999999999,"15min":0.96699999999999997},"min":{"1min":0.56699999999999995,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.444,"5min":1.623,"15min":1.623},"last":1.5780000000000001}]},{"osd":7,"last update":"Sun Sep 15 03:40:43 2024","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.121,"15min":1.0149999999999999},"min":{"1min":0.40500000000000003,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.4279999999999999,"5min":10.923999999999999,"15min":10.923999999999999},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.1160000000000001,"15min":1.01},"min":{"1min":0.52100000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.7130000000000001,"5min":10.74,"15min":10.74},"last":1.0960000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542195,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1179920,"kb_used_data":874248,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":93191920,"statfs":{"total":96636764160,"available":95428526080,"internally_reserved":0,"allocated":895229952,"data_stored":894483388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 03:39:45 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.68300000000000005,"15min":0.65900000000000003},"min":{"1min":0.30399999999999999,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.80800000000000005,"15min":0.83999999999999997},"min":{"1min":0.30199999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.139,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":0.81299999999999994}]},{"osd":1,"last update":"Sun Sep 15 03:39:45 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.746,"15min":0.70399999999999996},"min":{"1min":0.32900000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.1259999999999999,"5min":2.302,"15min":2.302},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.71099999999999997,"15min":0.68899999999999995},"min":{"1min":0.44600000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.23,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.90000000000000002}]},{"osd":2,"last update":"Sun Sep 15 03:39:45 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.82199999999999995,"15min":0.82399999999999995},"min":{"1min":0.54500000000000004,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.0800000000000001,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85399999999999998,"15min":0.85999999999999999},"min":{"1min":0.38300000000000001,"5min":0.314,"15min":0.314},"max":{"1min":1.3839999999999999,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":1.0249999999999999}]},{"osd":3,"last update":"Sun Sep 15 03:39:45 2024","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":1.669,"15min":2.0310000000000001},"min":{"1min":0.46700000000000003,"5min":0.375,"15min":0.375},"max":{"1min":1.2629999999999999,"5min":26.664999999999999,"15min":26.664999999999999},"last":1.073},{"interface":"front","average":{"1min":0.98199999999999998,"5min":1.827,"15min":2.2189999999999999},"min":{"1min":0.49399999999999999,"5min":0.312,"15min":0.312},"max":{"1min":1.5069999999999999,"5min":26.748000000000001,"15min":26.748000000000001},"last":1.111}]},{"osd":4,"last update":"Sun Sep 15 03:39:45 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.90000000000000002,"15min":0.88},"min":{"1min":0.34999999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.4510000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":1.1830000000000001},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.90500000000000003,"15min":0.875},"min":{"1min":0.47799999999999998,"5min":0.318,"15min":0.318},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":1.222}]},{"osd":6,"last update":"Sun Sep 15 03:39:45 2024","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":0.97499999999999998,"15min":0.95999999999999996},"min":{"1min":0.55700000000000005,"5min":0.373,"15min":0.373},"max":{"1min":1.6020000000000001,"5min":1.9730000000000001,"15min":1.9730000000000001},"last":0.97699999999999998},{"interface":"front","average":{"1min":1.121,"5min":1.026,"15min":1},"min":{"1min":0.439,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.5369999999999999,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.2689999999999999}]},{"osd":7,"last update":"Sun Sep 15 03:39:45 2024","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.94299999999999995,"15min":0.90200000000000002},"min":{"1min":0.42199999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.4470000000000001,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":1.29},{"interface":"front","average":{"1min":1.252,"5min":1.1020000000000001,"15min":1.071},"min":{"1min":0.54400000000000004,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":4.3070000000000004,"5min":4.3070000000000004,"15min":4.3070000000000004},"last":1.3400000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542195,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1279268,"kb_used_data":981916,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93092572,"statfs":{"total":96636764160,"available":95326793728,"internally_reserved":0,"allocated":1005481984,"data_stored":1004653769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 03:39:41 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.64700000000000002,"15min":0.69899999999999995},"min":{"1min":0.27400000000000002,"5min":0.22,"15min":0.22},"max":{"1min":1.0509999999999999,"5min":4.7809999999999997,"15min":4.7809999999999997},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.85899999999999999,"15min":0.99099999999999999},"min":{"1min":0.314,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.98899999999999999,"5min":7.7370000000000001,"15min":7.7370000000000001},"last":0.95899999999999996}]},{"osd":1,"last update":"Sun Sep 15 03:39:41 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.746,"15min":0.627},"min":{"1min":0.27600000000000002,"5min":0.217,"15min":0.217},"max":{"1min":1.002,"5min":13.749000000000001,"15min":13.749000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.68000000000000005,"5min":1.29,"15min":0.96599999999999997},"min":{"1min":0.31900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.052,"5min":46.225999999999999,"15min":46.225999999999999},"last":1.1419999999999999}]},{"osd":2,"last update":"Sun Sep 15 03:39:41 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.71699999999999997,"15min":0.69899999999999995},"min":{"1min":0.39100000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.1379999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.72799999999999998,"15min":0.71499999999999997},"min":{"1min":0.45400000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":1.0760000000000001}]},{"osd":4,"last update":"Sun Sep 15 03:39:41 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.92600000000000005,"15min":0.94199999999999995},"min":{"1min":0.40799999999999997,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.335,"5min":2.9870000000000001,"15min":2.9870000000000001},"last":0.877},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.94299999999999995,"15min":0.96499999999999997},"min":{"1min":0.42599999999999999,"5min":0.376,"15min":0.376},"max":{"1min":1.3080000000000001,"5min":2.609,"15min":2.609},"last":1.3109999999999999}]},{"osd":5,"last update":"Sun Sep 15 03:39:41 2024","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.86599999999999999,"15min":0.84099999999999997},"min":{"1min":0.47999999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.22,"5min":1.381,"15min":1.381},"last":1.52},{"interface":"front","average":{"1min":0.86099999999999999,"5min":1.0649999999999999,"15min":1.1040000000000001},"min":{"1min":0.40100000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.242,"5min":5.0309999999999997,"15min":5.0309999999999997},"last":1.284}]},{"osd":6,"last update":"Sun Sep 15 03:39:41 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.97599999999999998,"15min":0.97299999999999998},"min":{"1min":0.54600000000000004,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.3200000000000001,"5min":1.516,"15min":1.516},"last":1.3899999999999999},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.97499999999999998,"15min":0.96299999999999997},"min":{"1min":0.59699999999999998,"5min":0.503,"15min":0.503},"max":{"1min":1.5680000000000001,"5min":2.5550000000000002,"15min":2.5550000000000002},"last":1.115}]},{"osd":7,"last update":"Sun Sep 15 03:39:41 2024","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":1.1359999999999999,"15min":1.1850000000000001},"min":{"1min":0.57799999999999996,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.494,"5min":5.5439999999999996,"15min":5.5439999999999996},"last":1.036},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.399,"15min":1.5600000000000001},"min":{"1min":0.56999999999999995,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.357,"5min":9.5850000000000009,"15min":9.5850000000000009},"last":1.466}]}]},{"osd":2,"up_from":14,"seq":60129542197,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":683092,"kb_used_data":377416,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":93688748,"statfs":{"total":96636764160,"available":95937277952,"internally_reserved":0,"allocated":386473984,"data_stored":386097365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 03:40:46 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.59699999999999998,"15min":0.61199999999999999},"min":{"1min":0.39800000000000002,"5min":0.193,"15min":0.193},"max":{"1min":0.96499999999999997,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.66700000000000004,"15min":0.65700000000000003},"min":{"1min":0.54000000000000004,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":1.228,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.59299999999999997}]},{"osd":1,"last update":"Sun Sep 15 03:40:46 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.73099999999999998,"15min":0.72599999999999998},"min":{"1min":0.47299999999999998,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.044,"5min":1.258,"15min":1.258},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.61599999999999999,"15min":0.57999999999999996},"min":{"1min":0.41999999999999998,"5min":0.222,"15min":0.222},"max":{"1min":0.92800000000000005,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.71599999999999997}]},{"osd":3,"last update":"Sun Sep 15 03:40:46 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.68300000000000005,"15min":0.66000000000000003},"min":{"1min":0.42399999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.98299999999999998,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.77800000000000002,"15min":0.73399999999999999},"min":{"1min":0.40100000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.753}]},{"osd":4,"last update":"Sun Sep 15 03:40:46 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.84099999999999997,"15min":0.77900000000000003},"min":{"1min":0.55500000000000005,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.02,"5min":0.91100000000000003,"15min":0.86599999999999999},"min":{"1min":0.53100000000000003,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.4239999999999999,"5min":1.573,"15min":1.573},"last":0.92000000000000004}]},{"osd":5,"last update":"Sun Sep 15 03:40:46 2024","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.94299999999999995,"15min":0.95399999999999996},"min":{"1min":0.68500000000000005,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.3680000000000001,"5min":4.5839999999999996,"15min":4.5839999999999996},"last":0.95299999999999996},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0549999999999999,"15min":1.113},"min":{"1min":0.50900000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.341,"5min":4.8230000000000004,"15min":4.8230000000000004},"last":1.2430000000000001}]},{"osd":6,"last update":"Sun Sep 15 03:40:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":0.94199999999999995,"15min":0.86799999999999999},"min":{"1min":0.81100000000000005,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.288,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":0.88700000000000001},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.03,"15min":0.96999999999999997},"min":{"1min":0.49299999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.425,"5min":1.9039999999999999,"15min":1.9039999999999999},"last":1.0900000000000001}]},{"osd":7,"last update":"Sun Sep 15 03:40:46 2024","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.0389999999999999,"15min":1.044},"min":{"1min":0.72599999999999998,"5min":0.379,"15min":0.379},"max":{"1min":1.4590000000000001,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":1.331},{"interface":"front","average":{"1min":1.169,"5min":1.3540000000000001,"15min":1.6080000000000001},"min":{"1min":0.74099999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.4770000000000001,"5min":10.428000000000001,"15min":10.428000000000001},"last":1.022}]}]},{"osd":4,"up_from":13,"seq":55834574901,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1618136,"kb_used_data":1320784,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":92753704,"statfs":{"total":96636764160,"available":94979792896,"internally_reserved":0,"allocated":1352482816,"data_stored":1351530757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 03:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.85799999999999998,"15min":0.93999999999999995},"min":{"1min":0.311,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.137,"5min":6.5010000000000003,"15min":6.5010000000000003},"last":1.056},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.85299999999999998,"15min":0.89300000000000002},"min":{"1min":0.34300000000000003,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.159,"5min":3.645,"15min":3.645},"last":0.80100000000000005}]},{"osd":1,"last update":"Sun Sep 15 03:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.84499999999999997,"15min":0.88900000000000001},"min":{"1min":0.29299999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.2589999999999999,"5min":3.7069999999999999,"15min":3.7069999999999999},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.81699999999999995,"15min":0.80200000000000005},"min":{"1min":0.44600000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.272,"5min":2.5779999999999998,"15min":2.5779999999999998},"last":1.093}]},{"osd":2,"last update":"Sun Sep 15 03:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.84999999999999998,"15min":0.81100000000000005},"min":{"1min":0.45000000000000001,"5min":0.311,"15min":0.311},"max":{"1min":1.212,"5min":2.766,"15min":2.766},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.84199999999999997,"15min":0.77200000000000002},"min":{"1min":0.375,"5min":0.316,"15min":0.316},"max":{"1min":1.256,"5min":3.2749999999999999,"15min":3.2749999999999999},"last":0.90600000000000003}]},{"osd":3,"last update":"Sun Sep 15 03:40:45 2024","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.0609999999999999,"15min":1.173},"min":{"1min":0.501,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.357,"5min":6.3620000000000001,"15min":6.3620000000000001},"last":0.97699999999999998},{"interface":"front","average":{"1min":1.0640000000000001,"5min":0.94899999999999995,"15min":0.87},"min":{"1min":0.58799999999999997,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.4330000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":1.1759999999999999}]},{"osd":5,"last update":"Sun Sep 15 03:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":1.0069999999999999,"15min":1.0629999999999999},"min":{"1min":0.41899999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.6890000000000001,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":1.127},{"interface":"front","average":{"1min":1.171,"5min":1.143,"15min":1.107},"min":{"1min":0.53000000000000003,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.651,"5min":4.1710000000000003,"15min":4.1710000000000003},"last":1.3200000000000001}]},{"osd":6,"last update":"Sun Sep 15 03:40:45 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.0309999999999999,"15min":1.0389999999999999},"min":{"1min":0.45800000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.6120000000000001,"5min":3.669,"15min":3.669},"last":1.1990000000000001},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.0609999999999999,"15min":0.96399999999999997},"min":{"1min":0.73799999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.581,"5min":2.5329999999999999,"15min":2.5329999999999999},"last":1.2749999999999999}]},{"osd":7,"last update":"Sun Sep 15 03:40:45 2024","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.018,"15min":0.92700000000000005},"min":{"1min":0.56599999999999995,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.498,"5min":3.238,"15min":3.238},"last":1.0269999999999999},{"interface":"front","average":{"1min":1.244,"5min":1.1299999999999999,"15min":1.109},"min":{"1min":0.68700000000000006,"5min":0.378,"15min":0.378},"max":{"1min":1.552,"5min":3.8260000000000001,"15min":3.8260000000000001},"last":1.226}]}]},{"osd":1,"up_from":13,"seq":55834574900,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":986284,"kb_used_data":676452,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":93385556,"statfs":{"total":96636764160,"available":95626809344,"internally_reserved":0,"allocated":692686848,"data_stored":692165832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 03:39:44 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.56100000000000005,"15min":0.54300000000000004},"min":{"1min":0.28699999999999998,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.94599999999999995,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.64200000000000002,"15min":0.59899999999999998},"min":{"1min":0.25700000000000001,"5min":0.187,"15min":0.187},"max":{"1min":6.6050000000000004,"5min":6.6050000000000004,"15min":6.6050000000000004},"last":0.77300000000000002}]},{"osd":2,"last update":"Sun Sep 15 03:39:44 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.621,"15min":0.625},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.171,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.746,"15min":0.73999999999999999},"min":{"1min":0.32300000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.2070000000000001,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":1.0580000000000001}]},{"osd":3,"last update":"Sun Sep 15 03:39:44 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.66800000000000004,"15min":0.63300000000000001},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.93200000000000005},{"interface":"front","average":{"1min":1.1459999999999999,"5min":0.80400000000000005,"15min":0.72499999999999998},"min":{"1min":0.59199999999999997,"5min":0.255,"15min":0.255},"max":{"1min":5.6950000000000003,"5min":5.6950000000000003,"15min":5.6950000000000003},"last":1.123}]},{"osd":4,"last update":"Sun Sep 15 03:39:44 2024","interfaces":[{"interface":"back","average":{"1min":1.498,"5min":0.99399999999999999,"15min":0.88800000000000001},"min":{"1min":0.38300000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":9.1080000000000005,"5min":9.1080000000000005,"15min":9.1080000000000005},"last":1.2729999999999999},{"interface":"front","average":{"1min":1.5980000000000001,"5min":1.0669999999999999,"15min":0.97299999999999998},"min":{"1min":0.53100000000000003,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":9.1440000000000001,"5min":9.1440000000000001,"15min":9.1440000000000001},"last":1.3740000000000001}]},{"osd":5,"last update":"Sun Sep 15 03:39:44 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.90700000000000003,"15min":0.89100000000000001},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.236,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.046,"5min":0.98899999999999999,"15min":0.96199999999999997},"min":{"1min":0.47699999999999998,"5min":0.314,"15min":0.314},"max":{"1min":1.4970000000000001,"5min":2.3580000000000001,"15min":2.3580000000000001},"last":1.4530000000000001}]},{"osd":6,"last update":"Sun Sep 15 03:39:44 2024","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":0.96899999999999997,"15min":0.92900000000000005},"min":{"1min":0.40699999999999997,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":6.4109999999999996,"5min":6.4109999999999996,"15min":6.4109999999999996},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.0289999999999999,"15min":0.999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":5.3170000000000002,"5min":5.3170000000000002,"15min":5.3170000000000002},"last":1.341}]},{"osd":7,"last update":"Sun Sep 15 03:39:44 2024","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.3340000000000001,"15min":1.45},"min":{"1min":0.46300000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":6.3730000000000002,"5min":12.821,"15min":12.821},"last":1.403},{"interface":"front","average":{"1min":1.222,"5min":1.089,"15min":1.0669999999999999},"min":{"1min":0.505,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":5.2530000000000001,"5min":5.2530000000000001,"15min":5.2530000000000001},"last":1.4890000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574901,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1362492,"kb_used_data":1056820,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":93009348,"statfs":{"total":96636764160,"available":95241572352,"internally_reserved":0,"allocated":1082183680,"data_stored":1081273737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Sep 15 03:40:41 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.59399999999999997,"15min":0.59999999999999998},"min":{"1min":0.20799999999999999,"5min":0.19,"15min":0.19},"max":{"1min":2.9279999999999999,"5min":2.9279999999999999,"15min":2.9279999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.58099999999999996,"15min":0.54400000000000004},"min":{"1min":0.41399999999999998,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.79500000000000004}]},{"osd":2,"last update":"Sun Sep 15 03:39:46 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.57499999999999996,"15min":0.54500000000000004},"min":{"1min":0.317,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.69499999999999995,"15min":0.69899999999999995},"min":{"1min":0.313,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.1839999999999999,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":0.93899999999999995}]},{"osd":3,"last update":"Sun Sep 15 03:39:46 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.70199999999999996,"15min":0.71799999999999997},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.135,"5min":1.603,"15min":1.603},"last":0.747},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.66700000000000004,"15min":0.64800000000000002},"min":{"1min":0.28499999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.1060000000000001,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.872}]},{"osd":4,"last update":"Sun Sep 15 03:40:41 2024","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.83099999999999996,"15min":0.80200000000000005},"min":{"1min":0.45900000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.1919999999999999,"5min":2.403,"15min":2.403},"last":1.036},{"interface":"front","average":{"1min":1.135,"5min":0.89700000000000002,"15min":0.83499999999999996},"min":{"1min":0.41299999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":3.1059999999999999,"5min":3.1059999999999999,"15min":3.1059999999999999},"last":1.0920000000000001}]},{"osd":5,"last update":"Sun Sep 15 03:39:46 2024","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.85399999999999998,"15min":0.81799999999999995},"min":{"1min":0.38100000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":1.002},{"interface":"front","average":{"1min":1.0169999999999999,"5min":0.91500000000000004,"15min":0.86699999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.5189999999999999,"5min":2.4319999999999999,"15min":2.4319999999999999},"last":1.3440000000000001}]},{"osd":6,"last update":"Sun Sep 15 03:39:46 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.88400000000000001,"15min":0.85799999999999998},"min":{"1min":0.53900000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.4550000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.02,"5min":1.006,"15min":1},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.425,"5min":2.5459999999999998,"15min":2.5459999999999998},"last":1.2689999999999999}]},{"osd":7,"last update":"Sun Sep 15 03:39:46 2024","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.242,"15min":1.3109999999999999},"min":{"1min":0.69699999999999995,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.4790000000000001,"5min":10.561999999999999,"15min":10.561999999999999},"last":1.179},{"interface":"front","average":{"1min":0.995,"5min":0.91300000000000003,"15min":0.88600000000000001},"min":{"1min":0.45700000000000002,"5min":0.377,"15min":0.377},"max":{"1min":1.4510000000000001,"5min":1.554,"15min":1.554},"last":1.1559999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1081413632,"data_stored":1080658160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":692445184,"data_stored":692079164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":386224128,"data_stored":386004840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1005244416,"data_stored":1004567120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1352245248,"data_stored":1351444108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":894988288,"data_stored":894396720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":357982208,"data_stored":357742880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1072160768,"data_stored":1071536856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T03:40:50.039 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T03:40:50.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.139+0000 7f5de816f700 1 Processor -- start 2024-09-15T03:40:50.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.139+0000 7f5de816f700 1 -- start start 2024-09-15T03:40:50.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.139+0000 7f5de816f700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f5de0143090 0x7f5de006e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:50.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.139+0000 7f5de816f700 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5de0142c00 con 0x7f5de0142610 2024-09-15T03:40:50.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.139+0000 7f5de816f700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5de006f140 con 0x7f5de006deb0 2024-09-15T03:40:50.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.139+0000 7f5de816f700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5de006f2d0 con 0x7f5de0143090 2024-09-15T03:40:50.145 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.139+0000 7f5de670c700 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f5de0142610 0x7f5de0141090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:32866/0 (socket says 172.21.15.110:32866) 2024-09-15T03:40:50.145 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.139+0000 7f5de670c700 1 -- 172.21.15.110:0/3519747096 learned_addr learned my addr 172.21.15.110:0/3519747096 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-15T03:40:50.145 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.139+0000 7f5de570a700 1 --2- 172.21.15.110:0/3519747096 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f5de0143090 0x7f5de006e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:50.145 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.139+0000 7f5de4f09700 1 -- 172.21.15.110:0/3519747096 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3899606190 0 0) 0x7f5de0142c00 con 0x7f5de0142610 2024-09-15T03:40:50.145 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.139+0000 7f5de4f09700 1 -- 172.21.15.110:0/3519747096 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5dc8003570 con 0x7f5de0142610 2024-09-15T03:40:50.145 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.139+0000 7f5de570a700 1 -- 172.21.15.110:0/3519747096 >> v1:172.21.15.177:6789/0 conn(0x7f5de006deb0 legacy=0x7f5de006beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:50.145 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.139+0000 7f5de570a700 1 -- 172.21.15.110:0/3519747096 >> v1:172.21.15.110:6789/0 conn(0x7f5de0142610 legacy=0x7f5de0141090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:50.145 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.139+0000 7f5de570a700 1 -- 172.21.15.110:0/3519747096 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5de006c5c0 con 0x7f5de0143090 2024-09-15T03:40:50.145 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.139+0000 7f5de570a700 1 --2- 172.21.15.110:0/3519747096 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f5de0143090 0x7f5de006e280 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8009e90 tx=0x7f5dd8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=727bf2ad05046aaf server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:50.146 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.139+0000 7f5de4f09700 1 -- 172.21.15.110:0/3519747096 <== mon.2 v2:172.21.15.110:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd800e070 con 0x7f5de0143090 2024-09-15T03:40:50.146 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.139+0000 7f5de4f09700 1 -- 172.21.15.110:0/3519747096 <== mon.2 v2:172.21.15.110:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dd800bc30 con 0x7f5de0143090 2024-09-15T03:40:50.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.139+0000 7f5de4f09700 1 -- 172.21.15.110:0/3519747096 <== mon.2 v2:172.21.15.110:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd8012430 con 0x7f5de0143090 2024-09-15T03:40:50.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.143+0000 7f5de816f700 1 -- 172.21.15.110:0/3519747096 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f5de0143090 msgr2=0x7f5de006e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:50.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.143+0000 7f5de816f700 1 --2- 172.21.15.110:0/3519747096 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f5de0143090 0x7f5de006e280 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8009e90 tx=0x7f5dd8009780 comp rx=0 tx=0).stop 2024-09-15T03:40:50.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.143+0000 7f5de816f700 1 -- 172.21.15.110:0/3519747096 shutdown_connections 2024-09-15T03:40:50.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.143+0000 7f5de816f700 1 --2- 172.21.15.110:0/3519747096 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f5de0143090 0x7f5de006e280 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:50.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.143+0000 7f5de816f700 1 -- 172.21.15.110:0/3519747096 >> 172.21.15.110:0/3519747096 conn(0x7f5de00b06d0 msgr2=0x7f5de00b0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:40:50.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.143+0000 7f5de816f700 1 -- 172.21.15.110:0/3519747096 shutdown_connections 2024-09-15T03:40:50.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.143+0000 7f5de816f700 1 -- 172.21.15.110:0/3519747096 wait complete. 2024-09-15T03:40:50.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.143+0000 7f5de816f700 1 Processor -- start 2024-09-15T03:40:50.149 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.143+0000 7f5de816f700 1 -- start start 2024-09-15T03:40:50.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.147+0000 7f5de816f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5de006deb0 0x7f5de00a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:50.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.147+0000 7f5de816f700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f5de0142610 0x7f5de0154f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:50.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.147+0000 7f5de816f700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5de0143090 0x7f5de00a5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:50.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.147+0000 7f5de816f700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de00a5660 con 0x7f5de0143090 2024-09-15T03:40:50.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.147+0000 7f5de816f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de01559c0 con 0x7f5de006deb0 2024-09-15T03:40:50.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.147+0000 7f5de816f700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5de0155b80 con 0x7f5de0142610 2024-09-15T03:40:50.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.147+0000 7f5de570a700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f5de0142610 0x7f5de0154f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:50.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.147+0000 7f5de5f0b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5de006deb0 0x7f5de00a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:50.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.147+0000 7f5de670c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5de0143090 0x7f5de00a5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:50.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.147+0000 7f5de5f0b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5de006deb0 0x7f5de00a4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.110:42178/0 (socket says 172.21.15.110:42178) 2024-09-15T03:40:50.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.147+0000 7f5de5f0b700 1 -- 172.21.15.110:0/395207255 learned_addr learned my addr 172.21.15.110:0/395207255 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:40:50.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.147+0000 7f5de570a700 1 -- 172.21.15.110:0/395207255 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5de006deb0 msgr2=0x7f5de00a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:50.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.147+0000 7f5de570a700 1 --2- 172.21.15.110:0/395207255 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5de006deb0 0x7f5de00a4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:50.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.147+0000 7f5de570a700 1 -- 172.21.15.110:0/395207255 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5de0143090 msgr2=0x7f5de00a5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:50.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.147+0000 7f5de570a700 1 --2- 172.21.15.110:0/395207255 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5de0143090 0x7f5de00a5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:50.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.147+0000 7f5de570a700 1 -- 172.21.15.110:0/395207255 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd8009a20 con 0x7f5de0142610 2024-09-15T03:40:50.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.147+0000 7f5de5f0b700 1 --2- 172.21.15.110:0/395207255 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5de006deb0 0x7f5de00a4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T03:40:50.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.147+0000 7f5de570a700 1 --2- 172.21.15.110:0/395207255 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f5de0142610 0x7f5de0154f00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8016000 tx=0x7f5dd8019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:50.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.147+0000 7f5dddffb700 1 -- 172.21.15.110:0/395207255 <== mon.2 v2:172.21.15.110:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd8012d20 con 0x7f5de0142610 2024-09-15T03:40:50.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.147+0000 7f5dddffb700 1 -- 172.21.15.110:0/395207255 <== mon.2 v2:172.21.15.110:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dd8027470 con 0x7f5de0142610 2024-09-15T03:40:50.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.147+0000 7f5dddffb700 1 -- 172.21.15.110:0/395207255 <== mon.2 v2:172.21.15.110:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd8012ea0 con 0x7f5de0142610 2024-09-15T03:40:50.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.147+0000 7f5de816f700 1 -- 172.21.15.110:0/395207255 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5de0143f40 con 0x7f5de0142610 2024-09-15T03:40:50.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.147+0000 7f5de816f700 1 -- 172.21.15.110:0/395207255 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5de00a6030 con 0x7f5de0142610 2024-09-15T03:40:50.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.151+0000 7f5de816f700 1 -- 172.21.15.110:0/395207255 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5de00a57d0 con 0x7f5de0142610 2024-09-15T03:40:50.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.151+0000 7f5dddffb700 1 -- 172.21.15.110:0/395207255 <== mon.2 v2:172.21.15.110:3301/0 4 ==== mgrmap(e 5) v1 ==== 85847+0+0 (secure 0 0 0) 0x7f5dd8024070 con 0x7f5de0142610 2024-09-15T03:40:50.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.151+0000 7f5dddffb700 1 --2- 172.21.15.110:0/395207255 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f5dcc067ab0 0x7f5dcc069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:50.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.151+0000 7f5dddffb700 1 -- 172.21.15.110:0/395207255 <== mon.2 v2:172.21.15.110:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5641+0+0 (secure 0 0 0) 0x7f5dd8087a60 con 0x7f5de0142610 2024-09-15T03:40:50.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.151+0000 7f5de5f0b700 1 --2- 172.21.15.110:0/395207255 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f5dcc067ab0 0x7f5dcc069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:50.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.151+0000 7f5de5f0b700 1 --2- 172.21.15.110:0/395207255 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f5dcc067ab0 0x7f5dcc069f60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4001e90 tx=0x7f5dd4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:50.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.159+0000 7f5dddffb700 1 -- 172.21.15.110:0/395207255 <== mon.2 v2:172.21.15.110:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f5dd8054ba0 con 0x7f5de0142610 2024-09-15T03:40:50.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.327+0000 7f5de816f700 1 -- 172.21.15.110:0/395207255 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5de00721b0 con 0x7f5de0142610 2024-09-15T03:40:50.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.327+0000 7f5dddffb700 1 -- 172.21.15.110:0/395207255 <== mon.2 v2:172.21.15.110:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+15365 (secure 0 0 0) 0x7f5dd8057c20 con 0x7f5de0142610 2024-09-15T03:40:50.331 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-15T03:40:50.331 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":24,"fsid":"d7bdbc6e-e9f0-40e5-921a-f7c68fa2d8c1","created":"2024-09-15T03:36:28.375926+0000","modified":"2024-09-15T03:40:48.246453+0000","last_up_change":"2024-09-15T03:36:37.618040+0000","last_in_change":"2024-09-15T03:36:30.385381+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T03:36:36.657583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-15T03:36:39.048914+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephsqlite","create_time":"2024-09-15T03:36:47.199176+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{}}],"osds":[{"osd":0,"uuid":"09d93c1d-8ec1-42e8-8b59-5e20958830d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":22352},{"type":"v1","addr":"172.21.15.110:6801","nonce":22352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":22352},{"type":"v1","addr":"172.21.15.110:6803","nonce":22352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":22352},{"type":"v1","addr":"172.21.15.110:6807","nonce":22352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":22352},{"type":"v1","addr":"172.21.15.110:6805","nonce":22352}]},"public_addr":"172.21.15.110:6801/22352","cluster_addr":"172.21.15.110:6803/22352","heartbeat_back_addr":"172.21.15.110:6807/22352","heartbeat_front_addr":"172.21.15.110:6805/22352","state":["exists","up"]},{"osd":1,"uuid":"378acca0-6784-42c1-a6f8-0ba39a287db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":22354},{"type":"v1","addr":"172.21.15.110:6809","nonce":22354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":22354},{"type":"v1","addr":"172.21.15.110:6811","nonce":22354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":22354},{"type":"v1","addr":"172.21.15.110:6815","nonce":22354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":22354},{"type":"v1","addr":"172.21.15.110:6813","nonce":22354}]},"public_addr":"172.21.15.110:6809/22354","cluster_addr":"172.21.15.110:6811/22354","heartbeat_back_addr":"172.21.15.110:6815/22354","heartbeat_front_addr":"172.21.15.110:6813/22354","state":["exists","up"]},{"osd":2,"uuid":"b1c2c060-985a-429c-bf05-ccfd5fdf3d8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":22356},{"type":"v1","addr":"172.21.15.110:6824","nonce":22356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":22356},{"type":"v1","addr":"172.21.15.110:6827","nonce":22356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":22356},{"type":"v1","addr":"172.21.15.110:6831","nonce":22356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":22356},{"type":"v1","addr":"172.21.15.110:6829","nonce":22356}]},"public_addr":"172.21.15.110:6824/22356","cluster_addr":"172.21.15.110:6827/22356","heartbeat_back_addr":"172.21.15.110:6831/22356","heartbeat_front_addr":"172.21.15.110:6829/22356","state":["exists","up"]},{"osd":3,"uuid":"aca55104-d76b-4719-8837-252c2e55d0cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":22355},{"type":"v1","addr":"172.21.15.110:6817","nonce":22355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":22355},{"type":"v1","addr":"172.21.15.110:6819","nonce":22355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6823","nonce":22355},{"type":"v1","addr":"172.21.15.110:6825","nonce":22355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":22355},{"type":"v1","addr":"172.21.15.110:6821","nonce":22355}]},"public_addr":"172.21.15.110:6817/22355","cluster_addr":"172.21.15.110:6819/22355","heartbeat_back_addr":"172.21.15.110:6825/22355","heartbeat_front_addr":"172.21.15.110:6821/22355","state":["exists","up"]},{"osd":4,"uuid":"7a87be3c-b6ff-43a4-8672-39d3006a0e2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":22425},{"type":"v1","addr":"172.21.15.177:6824","nonce":22425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":22425},{"type":"v1","addr":"172.21.15.177:6827","nonce":22425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":22425},{"type":"v1","addr":"172.21.15.177:6831","nonce":22425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":22425},{"type":"v1","addr":"172.21.15.177:6829","nonce":22425}]},"public_addr":"172.21.15.177:6824/22425","cluster_addr":"172.21.15.177:6827/22425","heartbeat_back_addr":"172.21.15.177:6831/22425","heartbeat_front_addr":"172.21.15.177:6829/22425","state":["exists","up"]},{"osd":5,"uuid":"a8f3a63d-45af-41fe-be33-2339d7b9ed96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":22424},{"type":"v1","addr":"172.21.15.177:6804","nonce":22424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":22424},{"type":"v1","addr":"172.21.15.177:6808","nonce":22424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":22424},{"type":"v1","addr":"172.21.15.177:6815","nonce":22424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":22424},{"type":"v1","addr":"172.21.15.177:6812","nonce":22424}]},"public_addr":"172.21.15.177:6804/22424","cluster_addr":"172.21.15.177:6808/22424","heartbeat_back_addr":"172.21.15.177:6815/22424","heartbeat_front_addr":"172.21.15.177:6812/22424","state":["exists","up"]},{"osd":6,"uuid":"a61b52dd-273d-47b4-bf92-0610cce86e1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":22428},{"type":"v1","addr":"172.21.15.177:6817","nonce":22428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":22428},{"type":"v1","addr":"172.21.15.177:6819","nonce":22428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6823","nonce":22428},{"type":"v1","addr":"172.21.15.177:6825","nonce":22428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":22428},{"type":"v1","addr":"172.21.15.177:6821","nonce":22428}]},"public_addr":"172.21.15.177:6817/22428","cluster_addr":"172.21.15.177:6819/22428","heartbeat_back_addr":"172.21.15.177:6825/22428","heartbeat_front_addr":"172.21.15.177:6821/22428","state":["exists","up"]},{"osd":7,"uuid":"1a7d5a9f-6b41-47e1-b0ef-45f4d638fd65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":22423},{"type":"v1","addr":"172.21.15.177:6801","nonce":22423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6803","nonce":22423},{"type":"v1","addr":"172.21.15.177:6805","nonce":22423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6811","nonce":22423},{"type":"v1","addr":"172.21.15.177:6813","nonce":22423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6807","nonce":22423},{"type":"v1","addr":"172.21.15.177:6809","nonce":22423}]},"public_addr":"172.21.15.177:6801/22423","cluster_addr":"172.21.15.177:6805/22423","heartbeat_back_addr":"172.21.15.177:6813/22423","heartbeat_front_addr":"172.21.15.177:6809/22423","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:36:35.106295+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:36:35.495899+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:36:35.902086+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:36:35.652162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:36:35.645490+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:36:35.658420+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:36:35.622386+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:36:35.604615+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.110:0/4006587460":"2024-09-15T04:40:47.640898+0000","172.21.15.110:0/3981642940":"2024-09-15T04:39:31.839305+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T03:40:50.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.331+0000 7f5de816f700 1 -- 172.21.15.110:0/395207255 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f5dcc067ab0 msgr2=0x7f5dcc069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:50.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.331+0000 7f5de816f700 1 --2- 172.21.15.110:0/395207255 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f5dcc067ab0 0x7f5dcc069f60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4001e90 tx=0x7f5dd4009040 comp rx=0 tx=0).stop 2024-09-15T03:40:50.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.331+0000 7f5de816f700 1 -- 172.21.15.110:0/395207255 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f5de0142610 msgr2=0x7f5de0154f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:50.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.331+0000 7f5de816f700 1 --2- 172.21.15.110:0/395207255 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f5de0142610 0x7f5de0154f00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8016000 tx=0x7f5dd8019000 comp rx=0 tx=0).stop 2024-09-15T03:40:50.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.331+0000 7f5de816f700 1 -- 172.21.15.110:0/395207255 shutdown_connections 2024-09-15T03:40:50.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.331+0000 7f5de816f700 1 --2- 172.21.15.110:0/395207255 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f5dcc067ab0 0x7f5dcc069f60 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:50.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.331+0000 7f5de816f700 1 --2- 172.21.15.110:0/395207255 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5de0143090 0x7f5de00a5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:50.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.331+0000 7f5de816f700 1 --2- 172.21.15.110:0/395207255 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f5de0142610 0x7f5de0154f00 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:50.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.331+0000 7f5de816f700 1 --2- 172.21.15.110:0/395207255 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5de006deb0 0x7f5de00a4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:50.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.331+0000 7f5de816f700 1 -- 172.21.15.110:0/395207255 >> 172.21.15.110:0/395207255 conn(0x7f5de00b06d0 msgr2=0x7f5de013f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:40:50.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.331+0000 7f5de816f700 1 -- 172.21.15.110:0/395207255 shutdown_connections 2024-09-15T03:40:50.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:50.331+0000 7f5de816f700 1 -- 172.21.15.110:0/395207255 wait complete. 2024-09-15T03:40:51.352 INFO:tasks.ceph:Scrubbing osd.0 2024-09-15T03:40:51.352 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-15T03:40:51.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.471+0000 7f26541df700 1 Processor -- start 2024-09-15T03:40:51.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.471+0000 7f26541df700 1 -- start start 2024-09-15T03:40:51.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.471+0000 7f26541df700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f264c0b5160 0x7f264c0aeb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:51.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.471+0000 7f26541df700 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f264c06cae0 con 0x7f264c13aeb0 2024-09-15T03:40:51.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.471+0000 7f26541df700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f264c06e0e0 con 0x7f264c06c690 2024-09-15T03:40:51.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.471+0000 7f26541df700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f264c0b5530 con 0x7f264c0b5160 2024-09-15T03:40:51.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.475+0000 7f2651f7b700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f264c0b5160 0x7f264c0aeb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:51.478 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.475+0000 7f265277c700 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f264c13aeb0 0x7f264c148790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:32876/0 (socket says 172.21.15.110:32876) 2024-09-15T03:40:51.478 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.475+0000 7f265277c700 1 -- 172.21.15.110:0/1834572868 learned_addr learned my addr 172.21.15.110:0/1834572868 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-15T03:40:51.478 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.475+0000 7f2650f79700 1 -- 172.21.15.110:0/1834572868 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2378166536 0 0) 0x7f2644001490 con 0x7f264c13aeb0 2024-09-15T03:40:51.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.475+0000 7f2650f79700 1 -- 172.21.15.110:0/1834572868 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2638003570 con 0x7f264c13aeb0 2024-09-15T03:40:51.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.475+0000 7f2651f7b700 1 -- 172.21.15.110:0/1834572868 >> v1:172.21.15.177:6789/0 conn(0x7f264c06c690 legacy=0x7f264c06e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:51.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.475+0000 7f2651f7b700 1 -- 172.21.15.110:0/1834572868 >> v1:172.21.15.110:6789/0 conn(0x7f264c13aeb0 legacy=0x7f264c148790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:51.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.475+0000 7f2651f7b700 1 -- 172.21.15.110:0/1834572868 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f264c06e330 con 0x7f264c0b5160 2024-09-15T03:40:51.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.475+0000 7f2651f7b700 1 --2- 172.21.15.110:0/1834572868 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f264c0b5160 0x7f264c0aeb50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f263c009910 tx=0x7f263c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=694fd07dc8c4310b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:51.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.475+0000 7f2650f79700 1 -- 172.21.15.110:0/1834572868 <== mon.2 v2:172.21.15.110:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f263c00b680 con 0x7f264c0b5160 2024-09-15T03:40:51.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.475+0000 7f2650f79700 1 -- 172.21.15.110:0/1834572868 <== mon.2 v2:172.21.15.110:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f263c00b7e0 con 0x7f264c0b5160 2024-09-15T03:40:51.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.475+0000 7f2650f79700 1 -- 172.21.15.110:0/1834572868 <== mon.2 v2:172.21.15.110:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f263c00bba0 con 0x7f264c0b5160 2024-09-15T03:40:51.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.475+0000 7f26541df700 1 -- 172.21.15.110:0/1834572868 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f264c0b5160 msgr2=0x7f264c0aeb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:51.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.475+0000 7f26541df700 1 --2- 172.21.15.110:0/1834572868 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f264c0b5160 0x7f264c0aeb50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f263c009910 tx=0x7f263c0091a0 comp rx=0 tx=0).stop 2024-09-15T03:40:51.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.475+0000 7f26541df700 1 -- 172.21.15.110:0/1834572868 shutdown_connections 2024-09-15T03:40:51.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.475+0000 7f26541df700 1 --2- 172.21.15.110:0/1834572868 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f264c0b5160 0x7f264c0aeb50 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:51.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.475+0000 7f26541df700 1 -- 172.21.15.110:0/1834572868 >> 172.21.15.110:0/1834572868 conn(0x7f264c0680f0 msgr2=0x7f264c0b4820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:40:51.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.475+0000 7f26541df700 1 -- 172.21.15.110:0/1834572868 shutdown_connections 2024-09-15T03:40:51.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.475+0000 7f26541df700 1 -- 172.21.15.110:0/1834572868 wait complete. 2024-09-15T03:40:51.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.479+0000 7f26541df700 1 Processor -- start 2024-09-15T03:40:51.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.479+0000 7f26541df700 1 -- start start 2024-09-15T03:40:51.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.479+0000 7f26541df700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f264c06c690 0x7f264c151140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:51.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.479+0000 7f26541df700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f264c0b5160 0x7f264c151680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:51.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.479+0000 7f26541df700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f264c13aeb0 0x7f264c1365b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:51.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.479+0000 7f26541df700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f264c151d70 con 0x7f264c13aeb0 2024-09-15T03:40:51.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.479+0000 7f26541df700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f264c1420f0 con 0x7f264c0b5160 2024-09-15T03:40:51.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.479+0000 7f26541df700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f264c1422b0 con 0x7f264c06c690 2024-09-15T03:40:51.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.479+0000 7f2651f7b700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f264c06c690 0x7f264c151140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:51.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.479+0000 7f2651f7b700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f264c06c690 0x7f264c151140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3301/0 says I am v2:172.21.15.110:43054/0 (socket says 172.21.15.110:43054) 2024-09-15T03:40:51.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.479+0000 7f265277c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f264c13aeb0 0x7f264c1365b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:51.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.479+0000 7f2651f7b700 1 -- 172.21.15.110:0/47508689 learned_addr learned my addr 172.21.15.110:0/47508689 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:40:51.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.479+0000 7f265177a700 1 --2- 172.21.15.110:0/47508689 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f264c0b5160 0x7f264c151680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:51.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.479+0000 7f265277c700 1 -- 172.21.15.110:0/47508689 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f264c06c690 msgr2=0x7f264c151140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:51.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.479+0000 7f265277c700 1 --2- 172.21.15.110:0/47508689 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f264c06c690 0x7f264c151140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:51.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.479+0000 7f265277c700 1 -- 172.21.15.110:0/47508689 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f264c0b5160 msgr2=0x7f264c151680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:51.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.479+0000 7f265277c700 1 --2- 172.21.15.110:0/47508689 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f264c0b5160 0x7f264c151680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:51.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.479+0000 7f265277c700 1 -- 172.21.15.110:0/47508689 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f263c009420 con 0x7f264c13aeb0 2024-09-15T03:40:51.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.479+0000 7f265277c700 1 --2- 172.21.15.110:0/47508689 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f264c13aeb0 0x7f264c1365b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f264400cb10 tx=0x7f2644001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:51.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.479+0000 7f2641ffb700 1 -- 172.21.15.110:0/47508689 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f264400a7e0 con 0x7f264c13aeb0 2024-09-15T03:40:51.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.479+0000 7f2641ffb700 1 -- 172.21.15.110:0/47508689 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f264400a940 con 0x7f264c13aeb0 2024-09-15T03:40:51.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.479+0000 7f26541df700 1 -- 172.21.15.110:0/47508689 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f264c140a60 con 0x7f264c13aeb0 2024-09-15T03:40:51.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.479+0000 7f2641ffb700 1 -- 172.21.15.110:0/47508689 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f264400acc0 con 0x7f264c13aeb0 2024-09-15T03:40:51.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.479+0000 7f26541df700 1 -- 172.21.15.110:0/47508689 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f264c136c90 con 0x7f264c13aeb0 2024-09-15T03:40:51.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.483+0000 7f26541df700 1 -- 172.21.15.110:0/47508689 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f264c0721b0 con 0x7f264c13aeb0 2024-09-15T03:40:51.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.483+0000 7f2641ffb700 1 -- 172.21.15.110:0/47508689 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) v1 ==== 85847+0+0 (secure 0 0 0) 0x7f2644007300 con 0x7f264c13aeb0 2024-09-15T03:40:51.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.483+0000 7f2641ffb700 1 --2- 172.21.15.110:0/47508689 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f2630067a80 0x7f2630069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:51.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.483+0000 7f2651f7b700 1 --2- 172.21.15.110:0/47508689 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f2630067a80 0x7f2630069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:51.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.483+0000 7f2641ffb700 1 -- 172.21.15.110:0/47508689 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5641+0+0 (secure 0 0 0) 0x7f26440856e0 con 0x7f264c13aeb0 2024-09-15T03:40:51.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.483+0000 7f2641ffb700 1 --2- 172.21.15.110:0/47508689 >> [v2:172.21.15.110:6800/22352,v1:172.21.15.110:6801/22352] conn(0x7f263006ebd0 0x7f2630070fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:51.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.483+0000 7f2641ffb700 1 -- 172.21.15.110:0/47508689 --> [v2:172.21.15.110:6800/22352,v1:172.21.15.110:6801/22352] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2630071690 con 0x7f263006ebd0 2024-09-15T03:40:51.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.483+0000 7f2641ffb700 1 -- 172.21.15.110:0/47508689 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2644087d20 con 0x7f264c13aeb0 2024-09-15T03:40:51.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.483+0000 7f265177a700 1 --2- 172.21.15.110:0/47508689 >> [v2:172.21.15.110:6800/22352,v1:172.21.15.110:6801/22352] conn(0x7f263006ebd0 0x7f2630070fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:51.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.483+0000 7f2651f7b700 1 --2- 172.21.15.110:0/47508689 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f2630067a80 0x7f2630069f30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f263c00b7b0 tx=0x7f263c017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:51.490 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.483+0000 7f265177a700 1 --2- 172.21.15.110:0/47508689 >> [v2:172.21.15.110:6800/22352,v1:172.21.15.110:6801/22352] conn(0x7f263006ebd0 0x7f2630070fe0 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:51.490 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.487+0000 7f2641ffb700 1 -- 172.21.15.110:0/47508689 <== osd.0 v2:172.21.15.110:6800/22352 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f2630071690 con 0x7f263006ebd0 2024-09-15T03:40:51.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.503+0000 7f26541df700 1 -- 172.21.15.110:0/47508689 --> [v2:172.21.15.110:6800/22352,v1:172.21.15.110:6801/22352] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f264c06cae0 con 0x7f263006ebd0 2024-09-15T03:40:51.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.503+0000 7f2641ffb700 1 -- 172.21.15.110:0/47508689 <== osd.0 v2:172.21.15.110:6800/22352 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f264c06cae0 con 0x7f263006ebd0 2024-09-15T03:40:51.507 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-15T03:40:51.507 INFO:teuthology.orchestra.run.smithi110.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T03:40:51.507 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-15T03:40:51.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.503+0000 7f26541df700 1 -- 172.21.15.110:0/47508689 >> [v2:172.21.15.110:6800/22352,v1:172.21.15.110:6801/22352] conn(0x7f263006ebd0 msgr2=0x7f2630070fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:51.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.503+0000 7f26541df700 1 --2- 172.21.15.110:0/47508689 >> [v2:172.21.15.110:6800/22352,v1:172.21.15.110:6801/22352] conn(0x7f263006ebd0 0x7f2630070fe0 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:51.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.503+0000 7f26541df700 1 -- 172.21.15.110:0/47508689 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f2630067a80 msgr2=0x7f2630069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:51.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.503+0000 7f26541df700 1 --2- 172.21.15.110:0/47508689 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f2630067a80 0x7f2630069f30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f263c00b7b0 tx=0x7f263c017000 comp rx=0 tx=0).stop 2024-09-15T03:40:51.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.503+0000 7f26541df700 1 -- 172.21.15.110:0/47508689 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f264c13aeb0 msgr2=0x7f264c1365b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:51.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.503+0000 7f26541df700 1 --2- 172.21.15.110:0/47508689 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f264c13aeb0 0x7f264c1365b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f264400cb10 tx=0x7f2644001970 comp rx=0 tx=0).stop 2024-09-15T03:40:51.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.503+0000 7f265277c700 1 -- 172.21.15.110:0/47508689 reap_dead start 2024-09-15T03:40:51.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.503+0000 7f26541df700 1 -- 172.21.15.110:0/47508689 shutdown_connections 2024-09-15T03:40:51.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.503+0000 7f26541df700 1 -- 172.21.15.110:0/47508689 >> 172.21.15.110:0/47508689 conn(0x7f264c0680f0 msgr2=0x7f264c0af210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:40:51.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.503+0000 7f26541df700 1 -- 172.21.15.110:0/47508689 shutdown_connections 2024-09-15T03:40:51.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.503+0000 7f26541df700 1 -- 172.21.15.110:0/47508689 wait complete. 2024-09-15T03:40:51.520 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-15T03:40:51.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.627+0000 7f4864db4700 1 Processor -- start 2024-09-15T03:40:51.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.627+0000 7f4864db4700 1 -- start start 2024-09-15T03:40:51.633 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.627+0000 7f4864db4700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f48600b4db0 0x7f486013dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:51.633 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.627+0000 7f4864db4700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f48600b5180 0x7f4860141a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:51.633 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.627+0000 7f4864db4700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f486013e1b0 con 0x7f48600b06e0 2024-09-15T03:40:51.633 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.627+0000 7f4864db4700 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48600a8b10 con 0x7f48600b4db0 2024-09-15T03:40:51.633 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.627+0000 7f4864db4700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f486013daa0 con 0x7f48600b5180 2024-09-15T03:40:51.633 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.627+0000 7f485effd700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f48600b5180 0x7f4860141a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:51.633 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.627+0000 7f485f7fe700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f48600b4db0 0x7f486013dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:51.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.627+0000 7f485f7fe700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f48600b4db0 0x7f486013dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51098/0 (socket says 172.21.15.110:51098) 2024-09-15T03:40:51.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.627+0000 7f485f7fe700 1 -- 172.21.15.110:0/111067890 learned_addr learned my addr 172.21.15.110:0/111067890 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:40:51.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.627+0000 7f485effd700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f48600b5180 0x7f4860141a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3301/0 says I am v2:172.21.15.110:43068/0 (socket says 172.21.15.110:43068) 2024-09-15T03:40:51.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.627+0000 7f485effd700 1 -- 172.21.15.110:0/111067890 >> v2:172.21.15.110:3300/0 conn(0x7f48600b4db0 msgr2=0x7f486013dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:40:51.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.627+0000 7f485effd700 1 --2- 172.21.15.110:0/111067890 >> v2:172.21.15.110:3300/0 conn(0x7f48600b4db0 0x7f486013dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:51.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.627+0000 7f485effd700 1 -- 172.21.15.110:0/111067890 >> v1:172.21.15.177:6789/0 conn(0x7f48600b06e0 legacy=0x7f486006e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:51.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.627+0000 7f485e7fc700 1 -- 172.21.15.110:0/111067890 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1805680045 0 0) 0x7f486013e1b0 con 0x7f48600b06e0 2024-09-15T03:40:51.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.627+0000 7f485effd700 1 -- 172.21.15.110:0/111067890 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4860141200 con 0x7f48600b5180 2024-09-15T03:40:51.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.627+0000 7f485f7fe700 1 --2- 172.21.15.110:0/111067890 >> v2:172.21.15.110:3300/0 conn(0x7f48600b4db0 0x7f486013dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T03:40:51.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.627+0000 7f485effd700 1 --2- 172.21.15.110:0/111067890 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f48600b5180 0x7f4860141a00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4850009910 tx=0x7f48500091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3777c7ee5427386 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:51.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.627+0000 7f485e7fc700 1 -- 172.21.15.110:0/111067890 <== mon.2 v2:172.21.15.110:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f485000b640 con 0x7f48600b5180 2024-09-15T03:40:51.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.627+0000 7f485e7fc700 1 -- 172.21.15.110:0/111067890 <== mon.2 v2:172.21.15.110:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f485000b7a0 con 0x7f48600b5180 2024-09-15T03:40:51.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.627+0000 7f485e7fc700 1 -- 172.21.15.110:0/111067890 <== mon.2 v2:172.21.15.110:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f485000bb60 con 0x7f48600b5180 2024-09-15T03:40:51.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.631+0000 7f4864db4700 1 -- 172.21.15.110:0/111067890 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f48600b5180 msgr2=0x7f4860141a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:51.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.631+0000 7f4864db4700 1 --2- 172.21.15.110:0/111067890 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f48600b5180 0x7f4860141a00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4850009910 tx=0x7f48500091a0 comp rx=0 tx=0).stop 2024-09-15T03:40:51.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.631+0000 7f4864db4700 1 -- 172.21.15.110:0/111067890 shutdown_connections 2024-09-15T03:40:51.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.631+0000 7f4864db4700 1 --2- 172.21.15.110:0/111067890 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f48600b5180 0x7f4860141a00 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:51.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.631+0000 7f4864db4700 1 --2- 172.21.15.110:0/111067890 >> v2:172.21.15.110:3300/0 conn(0x7f48600b4db0 0x7f486013dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:51.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.631+0000 7f4864db4700 1 -- 172.21.15.110:0/111067890 >> 172.21.15.110:0/111067890 conn(0x7f48600680f0 msgr2=0x7f48600b46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:40:51.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.631+0000 7f4864db4700 1 -- 172.21.15.110:0/111067890 shutdown_connections 2024-09-15T03:40:51.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.631+0000 7f4864db4700 1 -- 172.21.15.110:0/111067890 wait complete. 2024-09-15T03:40:51.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.631+0000 7f4864db4700 1 Processor -- start 2024-09-15T03:40:51.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.631+0000 7f4864db4700 1 -- start start 2024-09-15T03:40:51.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.631+0000 7f4864db4700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f48600b06e0 0x7f48600b25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:51.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.631+0000 7f4864db4700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f48600b4db0 0x7f4860150a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:51.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.631+0000 7f4864db4700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f48600b5180 0x7f48600b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:51.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.631+0000 7f4864db4700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48600b2c50 con 0x7f48600b5180 2024-09-15T03:40:51.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.631+0000 7f4864db4700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4860151500 con 0x7f48600b06e0 2024-09-15T03:40:51.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.631+0000 7f4864db4700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48601516c0 con 0x7f48600b4db0 2024-09-15T03:40:51.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.631+0000 7f485effd700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f48600b4db0 0x7f4860150a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:51.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.631+0000 7f485ffff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f48600b5180 0x7f48600b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:51.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.631+0000 7f485effd700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f48600b4db0 0x7f4860150a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3301/0 says I am v2:172.21.15.110:43084/0 (socket says 172.21.15.110:43084) 2024-09-15T03:40:51.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.631+0000 7f485ffff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f48600b5180 0x7f48600b30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51114/0 (socket says 172.21.15.110:51114) 2024-09-15T03:40:51.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.631+0000 7f485effd700 1 -- 172.21.15.110:0/2786922397 learned_addr learned my addr 172.21.15.110:0/2786922397 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:40:51.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.631+0000 7f485f7fe700 1 --2- 172.21.15.110:0/2786922397 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f48600b06e0 0x7f48600b25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:51.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.635+0000 7f485ffff700 1 -- 172.21.15.110:0/2786922397 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f48600b4db0 msgr2=0x7f4860150a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:51.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.635+0000 7f485ffff700 1 --2- 172.21.15.110:0/2786922397 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f48600b4db0 0x7f4860150a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:51.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.635+0000 7f485ffff700 1 -- 172.21.15.110:0/2786922397 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f48600b06e0 msgr2=0x7f48600b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:51.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.635+0000 7f485ffff700 1 --2- 172.21.15.110:0/2786922397 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f48600b06e0 0x7f48600b25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:51.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.635+0000 7f485ffff700 1 -- 172.21.15.110:0/2786922397 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48540094f0 con 0x7f48600b5180 2024-09-15T03:40:51.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.635+0000 7f485ffff700 1 --2- 172.21.15.110:0/2786922397 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f48600b5180 0x7f48600b30e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f485800c890 tx=0x7f4858001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:51.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.635+0000 7f48477fe700 1 -- 172.21.15.110:0/2786922397 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4858012630 con 0x7f48600b5180 2024-09-15T03:40:51.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.635+0000 7f48477fe700 1 -- 172.21.15.110:0/2786922397 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48580127b0 con 0x7f48600b5180 2024-09-15T03:40:51.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.635+0000 7f4864db4700 1 -- 172.21.15.110:0/2786922397 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4850009420 con 0x7f48600b5180 2024-09-15T03:40:51.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.635+0000 7f48477fe700 1 -- 172.21.15.110:0/2786922397 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4858012b70 con 0x7f48600b5180 2024-09-15T03:40:51.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.635+0000 7f4864db4700 1 -- 172.21.15.110:0/2786922397 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48600b36a0 con 0x7f48600b5180 2024-09-15T03:40:51.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.635+0000 7f4864db4700 1 -- 172.21.15.110:0/2786922397 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48600b2e00 con 0x7f48600b5180 2024-09-15T03:40:51.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.639+0000 7f48477fe700 1 -- 172.21.15.110:0/2786922397 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) v1 ==== 85847+0+0 (secure 0 0 0) 0x7f4858003030 con 0x7f48600b5180 2024-09-15T03:40:51.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.639+0000 7f48477fe700 1 --2- 172.21.15.110:0/2786922397 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f484c067a80 0x7f484c069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:51.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.639+0000 7f48477fe700 1 -- 172.21.15.110:0/2786922397 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5641+0+0 (secure 0 0 0) 0x7f4858085500 con 0x7f48600b5180 2024-09-15T03:40:51.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.639+0000 7f485f7fe700 1 --2- 172.21.15.110:0/2786922397 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f484c067a80 0x7f484c069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:51.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.639+0000 7f485f7fe700 1 --2- 172.21.15.110:0/2786922397 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f484c067a80 0x7f484c069f30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4854006d20 tx=0x7f4854009250 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:51.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.647+0000 7f48477fe700 1 -- 172.21.15.110:0/2786922397 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f48600b2e00 con 0x7f48600b5180 2024-09-15T03:40:51.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.807+0000 7f4864db4700 1 -- 172.21.15.110:0/2786922397 --> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f48600b2e00 con 0x7f484c067a80 2024-09-15T03:40:51.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.807+0000 7f48477fe700 1 -- 172.21.15.110:0/2786922397 <== mgr.4104 v2:172.21.15.110:6832/22190 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f48600b2e00 con 0x7f484c067a80 2024-09-15T03:40:51.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.807+0000 7f4864db4700 1 -- 172.21.15.110:0/2786922397 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f484c067a80 msgr2=0x7f484c069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:51.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.807+0000 7f4864db4700 1 --2- 172.21.15.110:0/2786922397 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f484c067a80 0x7f484c069f30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4854006d20 tx=0x7f4854009250 comp rx=0 tx=0).stop 2024-09-15T03:40:51.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.807+0000 7f4864db4700 1 -- 172.21.15.110:0/2786922397 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f48600b5180 msgr2=0x7f48600b30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:51.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.807+0000 7f4864db4700 1 --2- 172.21.15.110:0/2786922397 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f48600b5180 0x7f48600b30e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f485800c890 tx=0x7f4858001630 comp rx=0 tx=0).stop 2024-09-15T03:40:51.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.807+0000 7f4864db4700 1 -- 172.21.15.110:0/2786922397 shutdown_connections 2024-09-15T03:40:51.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.807+0000 7f4864db4700 1 --2- 172.21.15.110:0/2786922397 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f484c067a80 0x7f484c069f30 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:51.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.807+0000 7f4864db4700 1 --2- 172.21.15.110:0/2786922397 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f48600b5180 0x7f48600b30e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:51.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.807+0000 7f4864db4700 1 --2- 172.21.15.110:0/2786922397 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f48600b4db0 0x7f4860150a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:51.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.807+0000 7f4864db4700 1 --2- 172.21.15.110:0/2786922397 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f48600b06e0 0x7f48600b25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:51.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.807+0000 7f4864db4700 1 -- 172.21.15.110:0/2786922397 >> 172.21.15.110:0/2786922397 conn(0x7f48600680f0 msgr2=0x7f486006bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:40:51.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.807+0000 7f4864db4700 1 -- 172.21.15.110:0/2786922397 shutdown_connections 2024-09-15T03:40:51.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.807+0000 7f4864db4700 1 -- 172.21.15.110:0/2786922397 wait complete. 2024-09-15T03:40:51.814 INFO:teuthology.orchestra.run.smithi110.stderr:instructed osd(s) 0 to deep-scrub 2024-09-15T03:40:51.824 INFO:tasks.ceph:Scrubbing osd.1 2024-09-15T03:40:51.824 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-15T03:40:51.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.931+0000 7fe4d92a5700 1 Processor -- start 2024-09-15T03:40:51.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.931+0000 7fe4d92a5700 1 -- start start 2024-09-15T03:40:51.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.931+0000 7fe4d92a5700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fe4d4063e40 0x7fe4d406e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:51.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.931+0000 7fe4d92a5700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fe4d4139dc0 0x7fe4d413a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:51.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.931+0000 7fe4d92a5700 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4d413a990 con 0x7fe4d406d7b0 2024-09-15T03:40:51.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.931+0000 7fe4d92a5700 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4d406f2e0 con 0x7fe4d4139dc0 2024-09-15T03:40:51.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.931+0000 7fe4d92a5700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d413a710 con 0x7fe4d4063e40 2024-09-15T03:40:51.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.931+0000 7fe4d37fe700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fe4d4139dc0 0x7fe4d413a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:51.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.931+0000 7fe4d3fff700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fe4d4063e40 0x7fe4d406e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:51.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.931+0000 7fe4d8aa4700 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fe4d406d7b0 0x7fe4d406dc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:32882/0 (socket says 172.21.15.110:32882) 2024-09-15T03:40:51.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.931+0000 7fe4d8aa4700 1 -- 172.21.15.110:0/1018798906 learned_addr learned my addr 172.21.15.110:0/1018798906 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-15T03:40:51.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.931+0000 7fe4d37fe700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fe4d4139dc0 0x7fe4d413a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51122/0 (socket says 172.21.15.110:51122) 2024-09-15T03:40:51.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.931+0000 7fe4d2ffd700 1 -- 172.21.15.110:0/1018798906 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1303944677 0 0) 0x7fe4d413a990 con 0x7fe4d406d7b0 2024-09-15T03:40:51.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.935+0000 7fe4d2ffd700 1 -- 172.21.15.110:0/1018798906 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4c0003570 con 0x7fe4d406d7b0 2024-09-15T03:40:51.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.935+0000 7fe4d3fff700 1 -- 172.21.15.110:0/1018798906 >> v2:172.21.15.110:3300/0 conn(0x7fe4d4139dc0 msgr2=0x7fe4d413a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:40:51.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.935+0000 7fe4d3fff700 1 --2- 172.21.15.110:0/1018798906 >> v2:172.21.15.110:3300/0 conn(0x7fe4d4139dc0 0x7fe4d413a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:51.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.935+0000 7fe4d3fff700 1 -- 172.21.15.110:0/1018798906 >> v1:172.21.15.110:6789/0 conn(0x7fe4d406d7b0 legacy=0x7fe4d406dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:51.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.935+0000 7fe4d3fff700 1 -- 172.21.15.110:0/1018798906 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4d40a5920 con 0x7fe4d4063e40 2024-09-15T03:40:51.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.935+0000 7fe4d3fff700 1 --2- 172.21.15.110:0/1018798906 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fe4d4063e40 0x7fe4d406e8b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc0098e0 tx=0x7fe4bc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=830695ba3e89100 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:51.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.935+0000 7fe4d2ffd700 1 -- 172.21.15.110:0/1018798906 <== mon.2 v2:172.21.15.110:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc011630 con 0x7fe4d4063e40 2024-09-15T03:40:51.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.935+0000 7fe4d2ffd700 1 -- 172.21.15.110:0/1018798906 <== mon.2 v2:172.21.15.110:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4bc011790 con 0x7fe4d4063e40 2024-09-15T03:40:51.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.935+0000 7fe4d2ffd700 1 -- 172.21.15.110:0/1018798906 <== mon.2 v2:172.21.15.110:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc011b30 con 0x7fe4d4063e40 2024-09-15T03:40:51.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.935+0000 7fe4d92a5700 1 -- 172.21.15.110:0/1018798906 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fe4d4063e40 msgr2=0x7fe4d406e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:51.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.935+0000 7fe4d92a5700 1 --2- 172.21.15.110:0/1018798906 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fe4d4063e40 0x7fe4d406e8b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc0098e0 tx=0x7fe4bc0091a0 comp rx=0 tx=0).stop 2024-09-15T03:40:51.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.935+0000 7fe4d92a5700 1 -- 172.21.15.110:0/1018798906 shutdown_connections 2024-09-15T03:40:51.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.935+0000 7fe4d92a5700 1 --2- 172.21.15.110:0/1018798906 >> v2:172.21.15.110:3300/0 conn(0x7fe4d4139dc0 0x7fe4d413a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:51.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.935+0000 7fe4d92a5700 1 --2- 172.21.15.110:0/1018798906 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fe4d4063e40 0x7fe4d406e8b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:51.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.935+0000 7fe4d92a5700 1 -- 172.21.15.110:0/1018798906 >> 172.21.15.110:0/1018798906 conn(0x7fe4d4069190 msgr2=0x7fe4d40b0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:40:51.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.935+0000 7fe4d92a5700 1 -- 172.21.15.110:0/1018798906 shutdown_connections 2024-09-15T03:40:51.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.935+0000 7fe4d92a5700 1 -- 172.21.15.110:0/1018798906 wait complete. 2024-09-15T03:40:51.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.935+0000 7fe4d92a5700 1 Processor -- start 2024-09-15T03:40:51.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.935+0000 7fe4d92a5700 1 -- start start 2024-09-15T03:40:51.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.935+0000 7fe4d92a5700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fe4d4063e40 0x7fe4d40b0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:51.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.935+0000 7fe4d92a5700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe4d406d7b0 0x7fe4d4150a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:51.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.935+0000 7fe4d92a5700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4d4139dc0 0x7fe4d4150f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:51.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.935+0000 7fe4d92a5700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d40aa6e0 con 0x7fe4d4139dc0 2024-09-15T03:40:51.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.935+0000 7fe4d92a5700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d40ab1e0 con 0x7fe4d406d7b0 2024-09-15T03:40:51.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.935+0000 7fe4d92a5700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d414d000 con 0x7fe4d4063e40 2024-09-15T03:40:51.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.935+0000 7fe4d3fff700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fe4d4063e40 0x7fe4d40b0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:51.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.935+0000 7fe4d3fff700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fe4d4063e40 0x7fe4d40b0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3301/0 says I am v2:172.21.15.110:43104/0 (socket says 172.21.15.110:43104) 2024-09-15T03:40:51.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.935+0000 7fe4d3fff700 1 -- 172.21.15.110:0/3859475193 learned_addr learned my addr 172.21.15.110:0/3859475193 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:40:51.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.935+0000 7fe4d8aa4700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4d4139dc0 0x7fe4d4150f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:51.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.935+0000 7fe4d37fe700 1 --2- 172.21.15.110:0/3859475193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe4d406d7b0 0x7fe4d4150a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:51.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.939+0000 7fe4d3fff700 1 -- 172.21.15.110:0/3859475193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe4d406d7b0 msgr2=0x7fe4d4150a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:51.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.939+0000 7fe4d3fff700 1 --2- 172.21.15.110:0/3859475193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe4d406d7b0 0x7fe4d4150a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:51.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.939+0000 7fe4d3fff700 1 -- 172.21.15.110:0/3859475193 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4d4139dc0 msgr2=0x7fe4d4150f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:51.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.939+0000 7fe4d3fff700 1 --2- 172.21.15.110:0/3859475193 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4d4139dc0 0x7fe4d4150f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:51.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.939+0000 7fe4d3fff700 1 -- 172.21.15.110:0/3859475193 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4bc009420 con 0x7fe4d4063e40 2024-09-15T03:40:51.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.939+0000 7fe4d37fe700 1 --2- 172.21.15.110:0/3859475193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe4d406d7b0 0x7fe4d4150a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T03:40:51.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.939+0000 7fe4d3fff700 1 --2- 172.21.15.110:0/3859475193 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fe4d4063e40 0x7fe4d40b0b80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc009ed0 tx=0x7fe4bc017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:51.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.939+0000 7fe4cbfff700 1 -- 172.21.15.110:0/3859475193 <== mon.2 v2:172.21.15.110:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc011900 con 0x7fe4d4063e40 2024-09-15T03:40:51.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.939+0000 7fe4cbfff700 1 -- 172.21.15.110:0/3859475193 <== mon.2 v2:172.21.15.110:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4bc011aa0 con 0x7fe4d4063e40 2024-09-15T03:40:51.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.939+0000 7fe4d92a5700 1 -- 172.21.15.110:0/3859475193 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4d414d580 con 0x7fe4d4063e40 2024-09-15T03:40:51.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.939+0000 7fe4cbfff700 1 -- 172.21.15.110:0/3859475193 <== mon.2 v2:172.21.15.110:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc025640 con 0x7fe4d4063e40 2024-09-15T03:40:51.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.939+0000 7fe4d92a5700 1 -- 172.21.15.110:0/3859475193 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4d414d9e0 con 0x7fe4d4063e40 2024-09-15T03:40:51.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.939+0000 7fe4d92a5700 1 -- 172.21.15.110:0/3859475193 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe4d40721b0 con 0x7fe4d4063e40 2024-09-15T03:40:51.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.939+0000 7fe4cbfff700 1 -- 172.21.15.110:0/3859475193 <== mon.2 v2:172.21.15.110:3301/0 4 ==== mgrmap(e 5) v1 ==== 85847+0+0 (secure 0 0 0) 0x7fe4bc025a30 con 0x7fe4d4063e40 2024-09-15T03:40:51.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.943+0000 7fe4cbfff700 1 --2- 172.21.15.110:0/3859475193 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fe4b4067a80 0x7fe4b4069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:51.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.943+0000 7fe4cbfff700 1 -- 172.21.15.110:0/3859475193 <== mon.2 v2:172.21.15.110:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5641+0+0 (secure 0 0 0) 0x7fe4bc087970 con 0x7fe4d4063e40 2024-09-15T03:40:51.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.943+0000 7fe4d37fe700 1 --2- 172.21.15.110:0/3859475193 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fe4b4067a80 0x7fe4b4069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:51.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.943+0000 7fe4cbfff700 1 --2- 172.21.15.110:0/3859475193 >> [v2:172.21.15.110:6808/22354,v1:172.21.15.110:6809/22354] conn(0x7fe4b406ebd0 0x7fe4b4070fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:51.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.943+0000 7fe4cbfff700 1 -- 172.21.15.110:0/3859475193 --> [v2:172.21.15.110:6808/22354,v1:172.21.15.110:6809/22354] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe4b4071690 con 0x7fe4b406ebd0 2024-09-15T03:40:51.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.943+0000 7fe4cbfff700 1 -- 172.21.15.110:0/3859475193 <== mon.2 v2:172.21.15.110:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4bc087c60 con 0x7fe4d4063e40 2024-09-15T03:40:51.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.943+0000 7fe4d8aa4700 1 --2- 172.21.15.110:0/3859475193 >> [v2:172.21.15.110:6808/22354,v1:172.21.15.110:6809/22354] conn(0x7fe4b406ebd0 0x7fe4b4070fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:51.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.943+0000 7fe4d37fe700 1 --2- 172.21.15.110:0/3859475193 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fe4b4067a80 0x7fe4b4069f30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe4d413c690 tx=0x7fe4c4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:51.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.943+0000 7fe4d8aa4700 1 --2- 172.21.15.110:0/3859475193 >> [v2:172.21.15.110:6808/22354,v1:172.21.15.110:6809/22354] conn(0x7fe4b406ebd0 0x7fe4b4070fe0 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:51.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.943+0000 7fe4cbfff700 1 -- 172.21.15.110:0/3859475193 <== osd.1 v2:172.21.15.110:6808/22354 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fe4b4071690 con 0x7fe4b406ebd0 2024-09-15T03:40:51.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.959+0000 7fe4d92a5700 1 -- 172.21.15.110:0/3859475193 --> [v2:172.21.15.110:6808/22354,v1:172.21.15.110:6809/22354] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fe4d414dea0 con 0x7fe4b406ebd0 2024-09-15T03:40:51.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.963+0000 7fe4cbfff700 1 -- 172.21.15.110:0/3859475193 <== osd.1 v2:172.21.15.110:6808/22354 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fe4d414dea0 con 0x7fe4b406ebd0 2024-09-15T03:40:51.966 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-15T03:40:51.966 INFO:teuthology.orchestra.run.smithi110.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T03:40:51.966 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-15T03:40:51.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.963+0000 7fe4d92a5700 1 -- 172.21.15.110:0/3859475193 >> [v2:172.21.15.110:6808/22354,v1:172.21.15.110:6809/22354] conn(0x7fe4b406ebd0 msgr2=0x7fe4b4070fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:51.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.963+0000 7fe4d92a5700 1 --2- 172.21.15.110:0/3859475193 >> [v2:172.21.15.110:6808/22354,v1:172.21.15.110:6809/22354] conn(0x7fe4b406ebd0 0x7fe4b4070fe0 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:51.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.963+0000 7fe4d92a5700 1 -- 172.21.15.110:0/3859475193 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fe4b4067a80 msgr2=0x7fe4b4069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:51.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.963+0000 7fe4d92a5700 1 --2- 172.21.15.110:0/3859475193 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fe4b4067a80 0x7fe4b4069f30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe4d413c690 tx=0x7fe4c4008040 comp rx=0 tx=0).stop 2024-09-15T03:40:51.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.963+0000 7fe4d92a5700 1 -- 172.21.15.110:0/3859475193 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fe4d4063e40 msgr2=0x7fe4d40b0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:51.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.963+0000 7fe4d92a5700 1 --2- 172.21.15.110:0/3859475193 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fe4d4063e40 0x7fe4d40b0b80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc009ed0 tx=0x7fe4bc017000 comp rx=0 tx=0).stop 2024-09-15T03:40:51.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.963+0000 7fe4d8aa4700 1 -- 172.21.15.110:0/3859475193 reap_dead start 2024-09-15T03:40:51.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.963+0000 7fe4d92a5700 1 -- 172.21.15.110:0/3859475193 shutdown_connections 2024-09-15T03:40:51.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.963+0000 7fe4d92a5700 1 -- 172.21.15.110:0/3859475193 >> 172.21.15.110:0/3859475193 conn(0x7fe4d4069190 msgr2=0x7fe4d413b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:40:51.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.963+0000 7fe4d92a5700 1 -- 172.21.15.110:0/3859475193 shutdown_connections 2024-09-15T03:40:51.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:51.963+0000 7fe4d92a5700 1 -- 172.21.15.110:0/3859475193 wait complete. 2024-09-15T03:40:51.985 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-15T03:40:52.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.095+0000 7f50a1be7700 1 Processor -- start 2024-09-15T03:40:52.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.095+0000 7f50a1be7700 1 -- start start 2024-09-15T03:40:52.101 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.095+0000 7f50a1be7700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f509c0b0e00 0x7f509c0a5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:52.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.095+0000 7f50a1be7700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f509c13f7d0 0x7f509c13b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:52.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.095+0000 7f50a1be7700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f509c13f010 con 0x7f509c0b0a30 2024-09-15T03:40:52.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.095+0000 7f50a1be7700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f509c0a6830 con 0x7f509c0b0e00 2024-09-15T03:40:52.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.095+0000 7f50a1be7700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f509c13f2b0 con 0x7f509c13f7d0 2024-09-15T03:40:52.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.095+0000 7f50a13e6700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f509c13f7d0 0x7f509c13b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:52.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.095+0000 7f50a13e6700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f509c13f7d0 0x7f509c13b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3301/0 says I am v2:172.21.15.110:43120/0 (socket says 172.21.15.110:43120) 2024-09-15T03:40:52.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.095+0000 7f50a13e6700 1 -- 172.21.15.110:0/2290716168 learned_addr learned my addr 172.21.15.110:0/2290716168 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:40:52.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.095+0000 7f5093fff700 1 --2- 172.21.15.110:0/2290716168 >> v2:172.21.15.177:3300/0 conn(0x7f509c0b0e00 0x7f509c0a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:52.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.095+0000 7f50a13e6700 1 -- 172.21.15.110:0/2290716168 >> v2:172.21.15.177:3300/0 conn(0x7f509c0b0e00 msgr2=0x7f509c0a5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:40:52.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.095+0000 7f50a13e6700 1 --2- 172.21.15.110:0/2290716168 >> v2:172.21.15.177:3300/0 conn(0x7f509c0b0e00 0x7f509c0a5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:52.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.095+0000 7f50a13e6700 1 -- 172.21.15.110:0/2290716168 >> v1:172.21.15.177:6789/0 conn(0x7f509c0b0a30 legacy=0x7f509c13bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:52.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.095+0000 7f50a13e6700 1 -- 172.21.15.110:0/2290716168 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f509c0acb70 con 0x7f509c13f7d0 2024-09-15T03:40:52.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.095+0000 7f5093fff700 1 --2- 172.21.15.110:0/2290716168 >> v2:172.21.15.177:3300/0 conn(0x7f509c0b0e00 0x7f509c0a5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T03:40:52.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.099+0000 7f50a13e6700 1 --2- 172.21.15.110:0/2290716168 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f509c13f7d0 0x7f509c13b140 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5094009f20 tx=0x7f5094009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39fef44e3ab7cab server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:52.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.099+0000 7f50937fe700 1 -- 172.21.15.110:0/2290716168 <== mon.2 v2:172.21.15.110:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509400e070 con 0x7f509c13f7d0 2024-09-15T03:40:52.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.099+0000 7f50937fe700 1 -- 172.21.15.110:0/2290716168 <== mon.2 v2:172.21.15.110:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f509400ba80 con 0x7f509c13f7d0 2024-09-15T03:40:52.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.099+0000 7f50937fe700 1 -- 172.21.15.110:0/2290716168 <== mon.2 v2:172.21.15.110:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50940092e0 con 0x7f509c13f7d0 2024-09-15T03:40:52.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.099+0000 7f50a1be7700 1 -- 172.21.15.110:0/2290716168 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f509c13f7d0 msgr2=0x7f509c13b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:52.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.099+0000 7f50a1be7700 1 --2- 172.21.15.110:0/2290716168 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f509c13f7d0 0x7f509c13b140 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5094009f20 tx=0x7f5094009840 comp rx=0 tx=0).stop 2024-09-15T03:40:52.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.099+0000 7f50a1be7700 1 -- 172.21.15.110:0/2290716168 shutdown_connections 2024-09-15T03:40:52.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.099+0000 7f50a1be7700 1 --2- 172.21.15.110:0/2290716168 >> v2:172.21.15.177:3300/0 conn(0x7f509c0b0e00 0x7f509c0a5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:52.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.099+0000 7f50a1be7700 1 --2- 172.21.15.110:0/2290716168 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f509c13f7d0 0x7f509c13b140 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:52.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.099+0000 7f50a1be7700 1 -- 172.21.15.110:0/2290716168 >> 172.21.15.110:0/2290716168 conn(0x7f509c0680f0 msgr2=0x7f509c0b0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:40:52.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.099+0000 7f50a1be7700 1 -- 172.21.15.110:0/2290716168 shutdown_connections 2024-09-15T03:40:52.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.099+0000 7f50a1be7700 1 -- 172.21.15.110:0/2290716168 wait complete. 2024-09-15T03:40:52.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.103+0000 7f50a1be7700 1 Processor -- start 2024-09-15T03:40:52.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.103+0000 7f50a1be7700 1 -- start start 2024-09-15T03:40:52.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.103+0000 7f50a1be7700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f509c0b0a30 0x7f509c0a7030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:52.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.103+0000 7f50a1be7700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f509c0b0e00 0x7f509c0a9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:52.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.103+0000 7f50a1be7700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f509c13f7d0 0x7f509c0a8650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:52.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.103+0000 7f50a1be7700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f509c0a8170 con 0x7f509c0b0e00 2024-09-15T03:40:52.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.103+0000 7f50a1be7700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f509c0a9a70 con 0x7f509c0b0a30 2024-09-15T03:40:52.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.103+0000 7f50a1be7700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f509c0a8b90 con 0x7f509c13f7d0 2024-09-15T03:40:52.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.103+0000 7f50a13e6700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f509c13f7d0 0x7f509c0a8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:52.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.103+0000 7f5093fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f509c0b0e00 0x7f509c0a9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:52.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.103+0000 7f50a13e6700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f509c13f7d0 0x7f509c0a8650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3301/0 says I am v2:172.21.15.110:43134/0 (socket says 172.21.15.110:43134) 2024-09-15T03:40:52.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.103+0000 7f50a13e6700 1 -- 172.21.15.110:0/1385942859 learned_addr learned my addr 172.21.15.110:0/1385942859 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:40:52.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.103+0000 7f50a0be5700 1 --2- 172.21.15.110:0/1385942859 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f509c0b0a30 0x7f509c0a7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:52.109 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.103+0000 7f50a13e6700 1 -- 172.21.15.110:0/1385942859 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f509c0b0a30 msgr2=0x7f509c0a7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:52.109 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.103+0000 7f50a13e6700 1 --2- 172.21.15.110:0/1385942859 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f509c0b0a30 0x7f509c0a7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:52.109 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.103+0000 7f50a13e6700 1 -- 172.21.15.110:0/1385942859 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f509c0b0e00 msgr2=0x7f509c0a9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:52.109 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.103+0000 7f50a13e6700 1 --2- 172.21.15.110:0/1385942859 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f509c0b0e00 0x7f509c0a9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:52.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.103+0000 7f50a13e6700 1 -- 172.21.15.110:0/1385942859 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5094009ae0 con 0x7f509c13f7d0 2024-09-15T03:40:52.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.103+0000 7f50a13e6700 1 --2- 172.21.15.110:0/1385942859 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f509c13f7d0 0x7f509c0a8650 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f509400b6c0 tx=0x7f5094018000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:52.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.103+0000 7f5090ff9700 1 -- 172.21.15.110:0/1385942859 <== mon.2 v2:172.21.15.110:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50940032f0 con 0x7f509c13f7d0 2024-09-15T03:40:52.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.103+0000 7f5090ff9700 1 -- 172.21.15.110:0/1385942859 <== mon.2 v2:172.21.15.110:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5094008d50 con 0x7f509c13f7d0 2024-09-15T03:40:52.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.103+0000 7f50a1be7700 1 -- 172.21.15.110:0/1385942859 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f509c142370 con 0x7f509c13f7d0 2024-09-15T03:40:52.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.103+0000 7f5090ff9700 1 -- 172.21.15.110:0/1385942859 <== mon.2 v2:172.21.15.110:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5094025690 con 0x7f509c13f7d0 2024-09-15T03:40:52.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.103+0000 7f50a1be7700 1 -- 172.21.15.110:0/1385942859 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f509c1382b0 con 0x7f509c13f7d0 2024-09-15T03:40:52.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.107+0000 7f50a1be7700 1 -- 172.21.15.110:0/1385942859 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f509c0ab3d0 con 0x7f509c13f7d0 2024-09-15T03:40:52.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.107+0000 7f5090ff9700 1 -- 172.21.15.110:0/1385942859 <== mon.2 v2:172.21.15.110:3301/0 4 ==== mgrmap(e 5) v1 ==== 85847+0+0 (secure 0 0 0) 0x7f5094003040 con 0x7f509c13f7d0 2024-09-15T03:40:52.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.107+0000 7f5090ff9700 1 --2- 172.21.15.110:0/1385942859 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f5088067a80 0x7f5088069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:52.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.107+0000 7f5090ff9700 1 -- 172.21.15.110:0/1385942859 <== mon.2 v2:172.21.15.110:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5641+0+0 (secure 0 0 0) 0x7f50940873f0 con 0x7f509c13f7d0 2024-09-15T03:40:52.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.107+0000 7f50a0be5700 1 --2- 172.21.15.110:0/1385942859 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f5088067a80 0x7f5088069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:52.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.107+0000 7f50a0be5700 1 --2- 172.21.15.110:0/1385942859 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f5088067a80 0x7f5088069f30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5098003a80 tx=0x7f5098007ec0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:52.119 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.115+0000 7f5090ff9700 1 -- 172.21.15.110:0/1385942859 <== mon.2 v2:172.21.15.110:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f509c0ab3d0 con 0x7f509c13f7d0 2024-09-15T03:40:52.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.279+0000 7f50a1be7700 1 -- 172.21.15.110:0/1385942859 --> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f509c0ab3d0 con 0x7f5088067a80 2024-09-15T03:40:52.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.279+0000 7f5090ff9700 1 -- 172.21.15.110:0/1385942859 <== mgr.4104 v2:172.21.15.110:6832/22190 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f509c0ab3d0 con 0x7f5088067a80 2024-09-15T03:40:52.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.279+0000 7f50a1be7700 1 -- 172.21.15.110:0/1385942859 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f5088067a80 msgr2=0x7f5088069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:52.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.279+0000 7f50a1be7700 1 --2- 172.21.15.110:0/1385942859 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f5088067a80 0x7f5088069f30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5098003a80 tx=0x7f5098007ec0 comp rx=0 tx=0).stop 2024-09-15T03:40:52.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.279+0000 7f50a1be7700 1 -- 172.21.15.110:0/1385942859 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f509c13f7d0 msgr2=0x7f509c0a8650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:52.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.279+0000 7f50a1be7700 1 --2- 172.21.15.110:0/1385942859 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f509c13f7d0 0x7f509c0a8650 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f509400b6c0 tx=0x7f5094018000 comp rx=0 tx=0).stop 2024-09-15T03:40:52.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.279+0000 7f50a1be7700 1 -- 172.21.15.110:0/1385942859 shutdown_connections 2024-09-15T03:40:52.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.279+0000 7f50a1be7700 1 --2- 172.21.15.110:0/1385942859 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f5088067a80 0x7f5088069f30 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:52.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.279+0000 7f50a1be7700 1 --2- 172.21.15.110:0/1385942859 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f509c13f7d0 0x7f509c0a8650 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:52.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.279+0000 7f50a1be7700 1 --2- 172.21.15.110:0/1385942859 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f509c0b0e00 0x7f509c0a9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:52.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.279+0000 7f50a1be7700 1 --2- 172.21.15.110:0/1385942859 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f509c0b0a30 0x7f509c0a7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:52.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.279+0000 7f50a1be7700 1 -- 172.21.15.110:0/1385942859 >> 172.21.15.110:0/1385942859 conn(0x7f509c0680f0 msgr2=0x7f509c0a4e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:40:52.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.279+0000 7f50a1be7700 1 -- 172.21.15.110:0/1385942859 shutdown_connections 2024-09-15T03:40:52.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.279+0000 7f50a1be7700 1 -- 172.21.15.110:0/1385942859 wait complete. 2024-09-15T03:40:52.286 INFO:teuthology.orchestra.run.smithi110.stderr:instructed osd(s) 1 to deep-scrub 2024-09-15T03:40:52.298 INFO:tasks.ceph:Scrubbing osd.2 2024-09-15T03:40:52.298 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-15T03:40:52.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.403+0000 7f1db274b700 1 Processor -- start 2024-09-15T03:40:52.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.403+0000 7f1db274b700 1 -- start start 2024-09-15T03:40:52.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.407+0000 7f1db274b700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f1dac063e40 0x7f1dac13a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:52.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.407+0000 7f1db274b700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f1dac06e230 0x7f1dac139ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:52.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.407+0000 7f1db274b700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dac0a5910 con 0x7f1dac06e820 2024-09-15T03:40:52.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.407+0000 7f1db274b700 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dac06d690 con 0x7f1dac06e230 2024-09-15T03:40:52.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.407+0000 7f1db274b700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dac064210 con 0x7f1dac063e40 2024-09-15T03:40:52.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.407+0000 7f1db1f4a700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f1dac06e230 0x7f1dac139ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:52.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.407+0000 7f1db1749700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f1dac063e40 0x7f1dac13a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:52.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.407+0000 7f1db1f4a700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f1dac06e230 0x7f1dac139ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51136/0 (socket says 172.21.15.110:51136) 2024-09-15T03:40:52.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.407+0000 7f1db1f4a700 1 -- 172.21.15.110:0/3440468392 learned_addr learned my addr 172.21.15.110:0/3440468392 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:40:52.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.407+0000 7f1db1f4a700 1 -- 172.21.15.110:0/3440468392 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f1dac063e40 msgr2=0x7f1dac13a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:40:52.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.407+0000 7f1db1f4a700 1 --2- 172.21.15.110:0/3440468392 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f1dac063e40 0x7f1dac13a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:52.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.407+0000 7f1db1f4a700 1 -- 172.21.15.110:0/3440468392 >> v1:172.21.15.177:6789/0 conn(0x7f1dac06e820 legacy=0x7f1dac06db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:52.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.407+0000 7f1db1f4a700 1 -- 172.21.15.110:0/3440468392 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dac06f1c0 con 0x7f1dac06e230 2024-09-15T03:40:52.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.407+0000 7f1db1f4a700 1 --2- 172.21.15.110:0/3440468392 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1dac06e230 0x7f1dac139ae0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1da400d1d0 tx=0x7f1da400ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc6b60fd89af23f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:52.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.407+0000 7f1da3fff700 1 -- 172.21.15.110:0/3440468392 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da4010070 con 0x7f1dac06e230 2024-09-15T03:40:52.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.407+0000 7f1da3fff700 1 -- 172.21.15.110:0/3440468392 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1da400de00 con 0x7f1dac06e230 2024-09-15T03:40:52.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.407+0000 7f1da3fff700 1 -- 172.21.15.110:0/3440468392 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da400b320 con 0x7f1dac06e230 2024-09-15T03:40:52.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.407+0000 7f1db274b700 1 -- 172.21.15.110:0/3440468392 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1dac06e230 msgr2=0x7f1dac139ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:52.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.407+0000 7f1db274b700 1 --2- 172.21.15.110:0/3440468392 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1dac06e230 0x7f1dac139ae0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1da400d1d0 tx=0x7f1da400ba50 comp rx=0 tx=0).stop 2024-09-15T03:40:52.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.407+0000 7f1db274b700 1 -- 172.21.15.110:0/3440468392 shutdown_connections 2024-09-15T03:40:52.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.407+0000 7f1db274b700 1 --2- 172.21.15.110:0/3440468392 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1dac06e230 0x7f1dac139ae0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:52.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.407+0000 7f1db274b700 1 --2- 172.21.15.110:0/3440468392 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f1dac063e40 0x7f1dac13a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:52.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.407+0000 7f1db274b700 1 -- 172.21.15.110:0/3440468392 >> 172.21.15.110:0/3440468392 conn(0x7f1dac069190 msgr2=0x7f1dac0b0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:40:52.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.407+0000 7f1db274b700 1 -- 172.21.15.110:0/3440468392 shutdown_connections 2024-09-15T03:40:52.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.407+0000 7f1db274b700 1 -- 172.21.15.110:0/3440468392 wait complete. 2024-09-15T03:40:52.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.411+0000 7f1db274b700 1 Processor -- start 2024-09-15T03:40:52.415 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.411+0000 7f1db274b700 1 -- start start 2024-09-15T03:40:52.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.411+0000 7f1db274b700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f1dac063e40 0x7f1dac0ac390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:52.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.411+0000 7f1db274b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1dac06e230 0x7f1dac138ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:52.417 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.411+0000 7f1db274b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1dac06e820 0x7f1dac139010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:52.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.411+0000 7f1db274b700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dac0abd50 con 0x7f1dac06e230 2024-09-15T03:40:52.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.411+0000 7f1db274b700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dac0aa7d0 con 0x7f1dac06e820 2024-09-15T03:40:52.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.411+0000 7f1db274b700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dac0ab830 con 0x7f1dac063e40 2024-09-15T03:40:52.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.411+0000 7f1db1749700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f1dac063e40 0x7f1dac0ac390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:52.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.411+0000 7f1db1749700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f1dac063e40 0x7f1dac0ac390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3301/0 says I am v2:172.21.15.110:43148/0 (socket says 172.21.15.110:43148) 2024-09-15T03:40:52.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.411+0000 7f1db1749700 1 -- 172.21.15.110:0/3901047055 learned_addr learned my addr 172.21.15.110:0/3901047055 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:40:52.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.411+0000 7f1db0f48700 1 --2- 172.21.15.110:0/3901047055 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1dac06e230 0x7f1dac138ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:52.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.411+0000 7f1db1f4a700 1 --2- 172.21.15.110:0/3901047055 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1dac06e820 0x7f1dac139010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:52.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.411+0000 7f1db1749700 1 -- 172.21.15.110:0/3901047055 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1dac06e820 msgr2=0x7f1dac139010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:52.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.411+0000 7f1db1749700 1 --2- 172.21.15.110:0/3901047055 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1dac06e820 0x7f1dac139010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:52.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.411+0000 7f1db1749700 1 -- 172.21.15.110:0/3901047055 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1dac06e230 msgr2=0x7f1dac138ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:52.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.411+0000 7f1db1749700 1 --2- 172.21.15.110:0/3901047055 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1dac06e230 0x7f1dac138ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:52.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.411+0000 7f1db1749700 1 -- 172.21.15.110:0/3901047055 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1da400bcf0 con 0x7f1dac063e40 2024-09-15T03:40:52.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.411+0000 7f1db1f4a700 1 --2- 172.21.15.110:0/3901047055 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1dac06e820 0x7f1dac139010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T03:40:52.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.411+0000 7f1db1749700 1 --2- 172.21.15.110:0/3901047055 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f1dac063e40 0x7f1dac0ac390 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1da800ecb0 tx=0x7f1da80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:52.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.415+0000 7f1da17fa700 1 -- 172.21.15.110:0/3901047055 <== mon.2 v2:172.21.15.110:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da800c920 con 0x7f1dac063e40 2024-09-15T03:40:52.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.415+0000 7f1da17fa700 1 -- 172.21.15.110:0/3901047055 <== mon.2 v2:172.21.15.110:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1da800ca80 con 0x7f1dac063e40 2024-09-15T03:40:52.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.415+0000 7f1db274b700 1 -- 172.21.15.110:0/3901047055 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dac139550 con 0x7f1dac063e40 2024-09-15T03:40:52.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.415+0000 7f1da17fa700 1 -- 172.21.15.110:0/3901047055 <== mon.2 v2:172.21.15.110:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da800ce40 con 0x7f1dac063e40 2024-09-15T03:40:52.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.415+0000 7f1db274b700 1 -- 172.21.15.110:0/3901047055 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dac139820 con 0x7f1dac063e40 2024-09-15T03:40:52.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.415+0000 7f1db274b700 1 -- 172.21.15.110:0/3901047055 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1dac0721b0 con 0x7f1dac063e40 2024-09-15T03:40:52.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.415+0000 7f1da17fa700 1 -- 172.21.15.110:0/3901047055 <== mon.2 v2:172.21.15.110:3301/0 4 ==== mgrmap(e 5) v1 ==== 85847+0+0 (secure 0 0 0) 0x7f1da8013680 con 0x7f1dac063e40 2024-09-15T03:40:52.420 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.415+0000 7f1da17fa700 1 --2- 172.21.15.110:0/3901047055 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f1d90067a80 0x7f1d90069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:52.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.415+0000 7f1da17fa700 1 -- 172.21.15.110:0/3901047055 <== mon.2 v2:172.21.15.110:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5641+0+0 (secure 0 0 0) 0x7f1da8085870 con 0x7f1dac063e40 2024-09-15T03:40:52.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.419+0000 7f1db0f48700 1 --2- 172.21.15.110:0/3901047055 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f1d90067a80 0x7f1d90069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:52.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.419+0000 7f1da17fa700 1 --2- 172.21.15.110:0/3901047055 >> [v2:172.21.15.110:6822/22356,v1:172.21.15.110:6824/22356] conn(0x7f1d9006ebd0 0x7f1d90070fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:52.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.419+0000 7f1da17fa700 1 -- 172.21.15.110:0/3901047055 --> [v2:172.21.15.110:6822/22356,v1:172.21.15.110:6824/22356] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1d90071690 con 0x7f1d9006ebd0 2024-09-15T03:40:52.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.419+0000 7f1da17fa700 1 -- 172.21.15.110:0/3901047055 <== mon.2 v2:172.21.15.110:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1da808a240 con 0x7f1dac063e40 2024-09-15T03:40:52.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.419+0000 7f1db1f4a700 1 --2- 172.21.15.110:0/3901047055 >> [v2:172.21.15.110:6822/22356,v1:172.21.15.110:6824/22356] conn(0x7f1d9006ebd0 0x7f1d90070fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:52.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.419+0000 7f1db0f48700 1 --2- 172.21.15.110:0/3901047055 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f1d90067a80 0x7f1d90069f30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1dac0ad730 tx=0x7f1d9c007ec0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:52.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.419+0000 7f1db1f4a700 1 --2- 172.21.15.110:0/3901047055 >> [v2:172.21.15.110:6822/22356,v1:172.21.15.110:6824/22356] conn(0x7f1d9006ebd0 0x7f1d90070fe0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:52.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.419+0000 7f1da17fa700 1 -- 172.21.15.110:0/3901047055 <== osd.2 v2:172.21.15.110:6822/22356 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f1d90071690 con 0x7f1d9006ebd0 2024-09-15T03:40:52.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.431+0000 7f1db274b700 1 -- 172.21.15.110:0/3901047055 --> [v2:172.21.15.110:6822/22356,v1:172.21.15.110:6824/22356] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f1dac0ab550 con 0x7f1d9006ebd0 2024-09-15T03:40:52.436 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.431+0000 7f1da17fa700 1 -- 172.21.15.110:0/3901047055 <== osd.2 v2:172.21.15.110:6822/22356 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f1dac0ab550 con 0x7f1d9006ebd0 2024-09-15T03:40:52.436 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-15T03:40:52.436 INFO:teuthology.orchestra.run.smithi110.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T03:40:52.436 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-15T03:40:52.437 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.431+0000 7f1db274b700 1 -- 172.21.15.110:0/3901047055 >> [v2:172.21.15.110:6822/22356,v1:172.21.15.110:6824/22356] conn(0x7f1d9006ebd0 msgr2=0x7f1d90070fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:52.437 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.431+0000 7f1db274b700 1 --2- 172.21.15.110:0/3901047055 >> [v2:172.21.15.110:6822/22356,v1:172.21.15.110:6824/22356] conn(0x7f1d9006ebd0 0x7f1d90070fe0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:52.437 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.431+0000 7f1db274b700 1 -- 172.21.15.110:0/3901047055 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f1d90067a80 msgr2=0x7f1d90069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:52.438 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.431+0000 7f1db274b700 1 --2- 172.21.15.110:0/3901047055 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f1d90067a80 0x7f1d90069f30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1dac0ad730 tx=0x7f1d9c007ec0 comp rx=0 tx=0).stop 2024-09-15T03:40:52.438 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.431+0000 7f1db274b700 1 -- 172.21.15.110:0/3901047055 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f1dac063e40 msgr2=0x7f1dac0ac390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:52.438 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.431+0000 7f1db274b700 1 --2- 172.21.15.110:0/3901047055 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f1dac063e40 0x7f1dac0ac390 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1da800ecb0 tx=0x7f1da80091a0 comp rx=0 tx=0).stop 2024-09-15T03:40:52.438 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.431+0000 7f1db1f4a700 1 -- 172.21.15.110:0/3901047055 reap_dead start 2024-09-15T03:40:52.438 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.431+0000 7f1db274b700 1 -- 172.21.15.110:0/3901047055 shutdown_connections 2024-09-15T03:40:52.438 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.431+0000 7f1db274b700 1 -- 172.21.15.110:0/3901047055 >> 172.21.15.110:0/3901047055 conn(0x7f1dac069190 msgr2=0x7f1dac13e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:40:52.438 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.431+0000 7f1db274b700 1 -- 172.21.15.110:0/3901047055 shutdown_connections 2024-09-15T03:40:52.438 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.431+0000 7f1db274b700 1 -- 172.21.15.110:0/3901047055 wait complete. 2024-09-15T03:40:52.448 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-15T03:40:52.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.547+0000 7f0cfdbd8700 1 Processor -- start 2024-09-15T03:40:52.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.547+0000 7f0cfdbd8700 1 -- start start 2024-09-15T03:40:52.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.551+0000 7f0cfdbd8700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f0cf80afc40 0x7f0cf813bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:52.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.551+0000 7f0cfdbd8700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f0cf80a59d0 0x7f0cf80ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:52.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.551+0000 7f0cfdbd8700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cf813f530 con 0x7f0cf80af870 2024-09-15T03:40:52.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.551+0000 7f0cfdbd8700 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cf813f150 con 0x7f0cf80afc40 2024-09-15T03:40:52.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.551+0000 7f0cfdbd8700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cf813f290 con 0x7f0cf80a59d0 2024-09-15T03:40:52.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.551+0000 7f0cf6ffd700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f0cf80afc40 0x7f0cf813bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:52.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.551+0000 7f0cf6ffd700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f0cf80afc40 0x7f0cf813bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51152/0 (socket says 172.21.15.110:51152) 2024-09-15T03:40:52.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.551+0000 7f0cf6ffd700 1 -- 172.21.15.110:0/1913619280 learned_addr learned my addr 172.21.15.110:0/1913619280 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:40:52.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.551+0000 7f0cf7fff700 1 --2- 172.21.15.110:0/1913619280 >> v2:172.21.15.177:3300/0 conn(0x7f0cf80a59d0 0x7f0cf80ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:52.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.551+0000 7f0cf6ffd700 1 -- 172.21.15.110:0/1913619280 >> v2:172.21.15.177:3300/0 conn(0x7f0cf80a59d0 msgr2=0x7f0cf80ae7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:40:52.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.551+0000 7f0cf6ffd700 1 --2- 172.21.15.110:0/1913619280 >> v2:172.21.15.177:3300/0 conn(0x7f0cf80a59d0 0x7f0cf80ae7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:52.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.551+0000 7f0cf6ffd700 1 -- 172.21.15.110:0/1913619280 >> v1:172.21.15.177:6789/0 conn(0x7f0cf80af870 legacy=0x7f0cf813fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:52.555 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.551+0000 7f0cf6ffd700 1 -- 172.21.15.110:0/1913619280 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cf813c120 con 0x7f0cf80afc40 2024-09-15T03:40:52.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.551+0000 7f0cf6ffd700 1 --2- 172.21.15.110:0/1913619280 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0cf80afc40 0x7f0cf813bbe0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0cf806fb50 tx=0x7f0cec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c9aee3eccb4e345 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:52.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.551+0000 7f0cf67fc700 1 -- 172.21.15.110:0/1913619280 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cec00b860 con 0x7f0cf80afc40 2024-09-15T03:40:52.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.551+0000 7f0cf67fc700 1 -- 172.21.15.110:0/1913619280 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cec00b9c0 con 0x7f0cf80afc40 2024-09-15T03:40:52.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.551+0000 7f0cf67fc700 1 -- 172.21.15.110:0/1913619280 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cec00bd80 con 0x7f0cf80afc40 2024-09-15T03:40:52.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.551+0000 7f0cfdbd8700 1 -- 172.21.15.110:0/1913619280 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0cf80afc40 msgr2=0x7f0cf813bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:52.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.551+0000 7f0cfdbd8700 1 --2- 172.21.15.110:0/1913619280 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0cf80afc40 0x7f0cf813bbe0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0cf806fb50 tx=0x7f0cec009250 comp rx=0 tx=0).stop 2024-09-15T03:40:52.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.551+0000 7f0cfdbd8700 1 -- 172.21.15.110:0/1913619280 shutdown_connections 2024-09-15T03:40:52.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.551+0000 7f0cfdbd8700 1 --2- 172.21.15.110:0/1913619280 >> v2:172.21.15.177:3300/0 conn(0x7f0cf80a59d0 0x7f0cf80ae7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:52.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.551+0000 7f0cfdbd8700 1 --2- 172.21.15.110:0/1913619280 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0cf80afc40 0x7f0cf813bbe0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:52.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.551+0000 7f0cfdbd8700 1 -- 172.21.15.110:0/1913619280 >> 172.21.15.110:0/1913619280 conn(0x7f0cf8067be0 msgr2=0x7f0cf8067fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:40:52.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.551+0000 7f0cfdbd8700 1 -- 172.21.15.110:0/1913619280 shutdown_connections 2024-09-15T03:40:52.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.551+0000 7f0cfdbd8700 1 -- 172.21.15.110:0/1913619280 wait complete. 2024-09-15T03:40:52.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.555+0000 7f0cfdbd8700 1 Processor -- start 2024-09-15T03:40:52.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.555+0000 7f0cfdbd8700 1 -- start start 2024-09-15T03:40:52.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.555+0000 7f0cfdbd8700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f0cf80a59d0 0x7f0cf80ab390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:52.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.555+0000 7f0cfdbd8700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0cf80af870 0x7f0cf80a7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:52.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.555+0000 7f0cfdbd8700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0cf80aa520 0x7f0cf814f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:52.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.555+0000 7f0cfdbd8700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf80a97f0 con 0x7f0cf80aa520 2024-09-15T03:40:52.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.555+0000 7f0cfdbd8700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf814f1d0 con 0x7f0cf80af870 2024-09-15T03:40:52.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.555+0000 7f0cfdbd8700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf814ee60 con 0x7f0cf80a59d0 2024-09-15T03:40:52.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.555+0000 7f0cf77fe700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f0cf80a59d0 0x7f0cf80ab390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:52.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.555+0000 7f0cf77fe700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f0cf80a59d0 0x7f0cf80ab390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3301/0 says I am v2:172.21.15.110:43162/0 (socket says 172.21.15.110:43162) 2024-09-15T03:40:52.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.555+0000 7f0cf7fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0cf80aa520 0x7f0cf814f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:52.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.555+0000 7f0cf77fe700 1 -- 172.21.15.110:0/2912442113 learned_addr learned my addr 172.21.15.110:0/2912442113 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:40:52.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.555+0000 7f0cf6ffd700 1 --2- 172.21.15.110:0/2912442113 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0cf80af870 0x7f0cf80a7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:52.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.555+0000 7f0cf77fe700 1 -- 172.21.15.110:0/2912442113 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0cf80af870 msgr2=0x7f0cf80a7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:52.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.555+0000 7f0cf77fe700 1 --2- 172.21.15.110:0/2912442113 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0cf80af870 0x7f0cf80a7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:52.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.555+0000 7f0cf77fe700 1 -- 172.21.15.110:0/2912442113 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0cf80aa520 msgr2=0x7f0cf814f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:52.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.555+0000 7f0cf77fe700 1 --2- 172.21.15.110:0/2912442113 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0cf80aa520 0x7f0cf814f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:52.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.555+0000 7f0cf77fe700 1 -- 172.21.15.110:0/2912442113 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cec0094f0 con 0x7f0cf80a59d0 2024-09-15T03:40:52.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.555+0000 7f0cf77fe700 1 --2- 172.21.15.110:0/2912442113 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f0cf80a59d0 0x7f0cf80ab390 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0cf000c910 tx=0x7f0cf0001710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:52.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.555+0000 7f0cdf7fe700 1 -- 172.21.15.110:0/2912442113 <== mon.2 v2:172.21.15.110:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf0006ba0 con 0x7f0cf80a59d0 2024-09-15T03:40:52.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.555+0000 7f0cdf7fe700 1 -- 172.21.15.110:0/2912442113 <== mon.2 v2:172.21.15.110:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cf0006d00 con 0x7f0cf80a59d0 2024-09-15T03:40:52.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.555+0000 7f0cfdbd8700 1 -- 172.21.15.110:0/2912442113 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cf80a7e80 con 0x7f0cf80a59d0 2024-09-15T03:40:52.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.555+0000 7f0cdf7fe700 1 -- 172.21.15.110:0/2912442113 <== mon.2 v2:172.21.15.110:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf0012880 con 0x7f0cf80a59d0 2024-09-15T03:40:52.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.555+0000 7f0cfdbd8700 1 -- 172.21.15.110:0/2912442113 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cf80aa9c0 con 0x7f0cf80a59d0 2024-09-15T03:40:52.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.555+0000 7f0cfdbd8700 1 -- 172.21.15.110:0/2912442113 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cf80721b0 con 0x7f0cf80a59d0 2024-09-15T03:40:52.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.559+0000 7f0cdf7fe700 1 -- 172.21.15.110:0/2912442113 <== mon.2 v2:172.21.15.110:3301/0 4 ==== mgrmap(e 5) v1 ==== 85847+0+0 (secure 0 0 0) 0x7f0cf0012c70 con 0x7f0cf80a59d0 2024-09-15T03:40:52.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.559+0000 7f0cdf7fe700 1 --2- 172.21.15.110:0/2912442113 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f0ce4067a80 0x7f0ce4069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:52.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.559+0000 7f0cf6ffd700 1 --2- 172.21.15.110:0/2912442113 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f0ce4067a80 0x7f0ce4069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:52.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.559+0000 7f0cdf7fe700 1 -- 172.21.15.110:0/2912442113 <== mon.2 v2:172.21.15.110:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5641+0+0 (secure 0 0 0) 0x7f0cf0085b80 con 0x7f0cf80a59d0 2024-09-15T03:40:52.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.559+0000 7f0cf6ffd700 1 --2- 172.21.15.110:0/2912442113 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f0ce4067a80 0x7f0ce4069f30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0cf806fb50 tx=0x7f0cec017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:52.568 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.563+0000 7f0cdf7fe700 1 -- 172.21.15.110:0/2912442113 <== mon.2 v2:172.21.15.110:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f0cf0052cc0 con 0x7f0cf80a59d0 2024-09-15T03:40:52.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.731+0000 7f0cfdbd8700 1 -- 172.21.15.110:0/2912442113 --> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f0cf814efa0 con 0x7f0ce4067a80 2024-09-15T03:40:52.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.731+0000 7f0cdf7fe700 1 -- 172.21.15.110:0/2912442113 <== mgr.4104 v2:172.21.15.110:6832/22190 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f0cf814efa0 con 0x7f0ce4067a80 2024-09-15T03:40:52.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.735+0000 7f0cfdbd8700 1 -- 172.21.15.110:0/2912442113 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f0ce4067a80 msgr2=0x7f0ce4069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:52.738 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.735+0000 7f0cfdbd8700 1 --2- 172.21.15.110:0/2912442113 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f0ce4067a80 0x7f0ce4069f30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0cf806fb50 tx=0x7f0cec017000 comp rx=0 tx=0).stop 2024-09-15T03:40:52.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.735+0000 7f0cfdbd8700 1 -- 172.21.15.110:0/2912442113 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f0cf80a59d0 msgr2=0x7f0cf80ab390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:52.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.735+0000 7f0cfdbd8700 1 --2- 172.21.15.110:0/2912442113 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f0cf80a59d0 0x7f0cf80ab390 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0cf000c910 tx=0x7f0cf0001710 comp rx=0 tx=0).stop 2024-09-15T03:40:52.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.735+0000 7f0cfdbd8700 1 -- 172.21.15.110:0/2912442113 shutdown_connections 2024-09-15T03:40:52.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.735+0000 7f0cfdbd8700 1 --2- 172.21.15.110:0/2912442113 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f0ce4067a80 0x7f0ce4069f30 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:52.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.735+0000 7f0cfdbd8700 1 --2- 172.21.15.110:0/2912442113 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0cf80aa520 0x7f0cf814f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:52.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.735+0000 7f0cfdbd8700 1 --2- 172.21.15.110:0/2912442113 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0cf80af870 0x7f0cf80a7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:52.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.735+0000 7f0cfdbd8700 1 --2- 172.21.15.110:0/2912442113 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f0cf80a59d0 0x7f0cf80ab390 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:52.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.735+0000 7f0cfdbd8700 1 -- 172.21.15.110:0/2912442113 >> 172.21.15.110:0/2912442113 conn(0x7f0cf8067be0 msgr2=0x7f0cf813a920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:40:52.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.735+0000 7f0cfdbd8700 1 -- 172.21.15.110:0/2912442113 shutdown_connections 2024-09-15T03:40:52.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.735+0000 7f0cfdbd8700 1 -- 172.21.15.110:0/2912442113 wait complete. 2024-09-15T03:40:52.740 INFO:teuthology.orchestra.run.smithi110.stderr:instructed osd(s) 2 to deep-scrub 2024-09-15T03:40:52.753 INFO:tasks.ceph:Scrubbing osd.3 2024-09-15T03:40:52.753 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-15T03:40:52.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.855+0000 7f71cad39700 1 Processor -- start 2024-09-15T03:40:52.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.855+0000 7f71cad39700 1 -- start start 2024-09-15T03:40:52.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.859+0000 7f71cad39700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f71c413e160 0x7f71c413e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:52.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.859+0000 7f71cad39700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f71c40a4b20 0x7f71c40ad770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:52.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.859+0000 7f71cad39700 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71c413bbe0 con 0x7f71c413ed60 2024-09-15T03:40:52.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.859+0000 7f71cad39700 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71c40a5aa0 con 0x7f71c40a4b20 2024-09-15T03:40:52.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.859+0000 7f71cad39700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71c406fb80 con 0x7f71c413e160 2024-09-15T03:40:52.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.859+0000 7f71c9536700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f71c40a4b20 0x7f71c40ad770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:52.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.859+0000 7f71ca538700 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f71c413ed60 0x7f71c406cdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:32884/0 (socket says 172.21.15.110:32884) 2024-09-15T03:40:52.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.859+0000 7f71ca538700 1 -- 172.21.15.110:0/687523469 learned_addr learned my addr 172.21.15.110:0/687523469 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-15T03:40:52.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.859+0000 7f71c9d37700 1 --2- 172.21.15.110:0/687523469 >> v2:172.21.15.177:3300/0 conn(0x7f71c413e160 0x7f71c413e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:52.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.859+0000 7f71c9536700 1 -- 172.21.15.110:0/687523469 >> v2:172.21.15.177:3300/0 conn(0x7f71c413e160 msgr2=0x7f71c413e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:40:52.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.859+0000 7f71c9536700 1 --2- 172.21.15.110:0/687523469 >> v2:172.21.15.177:3300/0 conn(0x7f71c413e160 0x7f71c413e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:52.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.859+0000 7f71c9536700 1 -- 172.21.15.110:0/687523469 >> v1:172.21.15.110:6789/0 conn(0x7f71c413ed60 legacy=0x7f71c406cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:52.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.859+0000 7f71c8d35700 1 -- 172.21.15.110:0/687523469 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1755348309 0 0) 0x7f71c413bbe0 con 0x7f71c413ed60 2024-09-15T03:40:52.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.859+0000 7f71c9536700 1 -- 172.21.15.110:0/687523469 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71c413a900 con 0x7f71c40a4b20 2024-09-15T03:40:52.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.859+0000 7f71c9536700 1 --2- 172.21.15.110:0/687523469 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71c40a4b20 0x7f71c40ad770 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f71c4064900 tx=0x7f71b40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eed12a24110ebd9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:52.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.859+0000 7f71c8d35700 1 -- 172.21.15.110:0/687523469 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b4011630 con 0x7f71c40a4b20 2024-09-15T03:40:52.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.859+0000 7f71c8d35700 1 -- 172.21.15.110:0/687523469 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71b4011790 con 0x7f71c40a4b20 2024-09-15T03:40:52.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.859+0000 7f71c8d35700 1 -- 172.21.15.110:0/687523469 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b4011b30 con 0x7f71c40a4b20 2024-09-15T03:40:52.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.859+0000 7f71cad39700 1 -- 172.21.15.110:0/687523469 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71c40a4b20 msgr2=0x7f71c40ad770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:52.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.859+0000 7f71cad39700 1 --2- 172.21.15.110:0/687523469 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71c40a4b20 0x7f71c40ad770 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f71c4064900 tx=0x7f71b40091a0 comp rx=0 tx=0).stop 2024-09-15T03:40:52.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.859+0000 7f71cad39700 1 -- 172.21.15.110:0/687523469 shutdown_connections 2024-09-15T03:40:52.865 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.859+0000 7f71cad39700 1 --2- 172.21.15.110:0/687523469 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71c40a4b20 0x7f71c40ad770 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:52.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.859+0000 7f71cad39700 1 --2- 172.21.15.110:0/687523469 >> v2:172.21.15.177:3300/0 conn(0x7f71c413e160 0x7f71c413e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:52.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.859+0000 7f71cad39700 1 -- 172.21.15.110:0/687523469 >> 172.21.15.110:0/687523469 conn(0x7f71c40b1500 msgr2=0x7f71c40b1900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:40:52.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.859+0000 7f71cad39700 1 -- 172.21.15.110:0/687523469 shutdown_connections 2024-09-15T03:40:52.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.859+0000 7f71cad39700 1 -- 172.21.15.110:0/687523469 wait complete. 2024-09-15T03:40:52.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.863+0000 7f71cad39700 1 Processor -- start 2024-09-15T03:40:52.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.863+0000 7f71cad39700 1 -- start start 2024-09-15T03:40:52.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.863+0000 7f71cad39700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f71c40a4b20 0x7f71c4134f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:52.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.863+0000 7f71cad39700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71c413e160 0x7f71c414db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:52.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.863+0000 7f71cad39700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f71c413ed60 0x7f71c413f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:52.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.863+0000 7f71cad39700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c414e3a0 con 0x7f71c413e160 2024-09-15T03:40:52.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.863+0000 7f71cad39700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c414e060 con 0x7f71c413ed60 2024-09-15T03:40:52.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.863+0000 7f71cad39700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71c4140590 con 0x7f71c40a4b20 2024-09-15T03:40:52.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.863+0000 7f71c9536700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71c413e160 0x7f71c414db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:52.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.863+0000 7f71c9d37700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f71c40a4b20 0x7f71c4134f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:52.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.863+0000 7f71c9d37700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f71c40a4b20 0x7f71c4134f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3301/0 says I am v2:172.21.15.110:43178/0 (socket says 172.21.15.110:43178) 2024-09-15T03:40:52.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.863+0000 7f71c9d37700 1 -- 172.21.15.110:0/1619930074 learned_addr learned my addr 172.21.15.110:0/1619930074 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:40:52.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.863+0000 7f71c9536700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71c413e160 0x7f71c414db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51186/0 (socket says 172.21.15.110:51186) 2024-09-15T03:40:52.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.863+0000 7f71ca538700 1 --2- 172.21.15.110:0/1619930074 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f71c413ed60 0x7f71c413f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:52.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.863+0000 7f71c9d37700 1 -- 172.21.15.110:0/1619930074 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f71c413ed60 msgr2=0x7f71c413f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:52.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.863+0000 7f71c9d37700 1 --2- 172.21.15.110:0/1619930074 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f71c413ed60 0x7f71c413f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:52.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.863+0000 7f71c9d37700 1 -- 172.21.15.110:0/1619930074 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71c413e160 msgr2=0x7f71c414db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:52.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.863+0000 7f71c9d37700 1 --2- 172.21.15.110:0/1619930074 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71c413e160 0x7f71c414db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:52.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.863+0000 7f71c9d37700 1 -- 172.21.15.110:0/1619930074 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71b4009420 con 0x7f71c40a4b20 2024-09-15T03:40:52.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.863+0000 7f71c9d37700 1 --2- 172.21.15.110:0/1619930074 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f71c40a4b20 0x7f71c4134f10 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f71bc009ef0 tx=0x7f71bc00b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:52.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.863+0000 7f71b9ffb700 1 -- 172.21.15.110:0/1619930074 <== mon.2 v2:172.21.15.110:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71bc00cc20 con 0x7f71c40a4b20 2024-09-15T03:40:52.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.867+0000 7f71b9ffb700 1 -- 172.21.15.110:0/1619930074 <== mon.2 v2:172.21.15.110:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71bc00f040 con 0x7f71c40a4b20 2024-09-15T03:40:52.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.867+0000 7f71cad39700 1 -- 172.21.15.110:0/1619930074 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71c40ad020 con 0x7f71c40a4b20 2024-09-15T03:40:52.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.867+0000 7f71b9ffb700 1 -- 172.21.15.110:0/1619930074 <== mon.2 v2:172.21.15.110:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71bc014480 con 0x7f71c40a4b20 2024-09-15T03:40:52.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.867+0000 7f71cad39700 1 -- 172.21.15.110:0/1619930074 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71c413f9c0 con 0x7f71c40a4b20 2024-09-15T03:40:52.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.867+0000 7f71cad39700 1 -- 172.21.15.110:0/1619930074 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f71c413fe50 con 0x7f71c40a4b20 2024-09-15T03:40:52.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.867+0000 7f71b9ffb700 1 -- 172.21.15.110:0/1619930074 <== mon.2 v2:172.21.15.110:3301/0 4 ==== mgrmap(e 5) v1 ==== 85847+0+0 (secure 0 0 0) 0x7f71bc0147f0 con 0x7f71c40a4b20 2024-09-15T03:40:52.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.871+0000 7f71b9ffb700 1 --2- 172.21.15.110:0/1619930074 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f71a8067a80 0x7f71a8069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:52.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.871+0000 7f71c9536700 1 --2- 172.21.15.110:0/1619930074 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f71a8067a80 0x7f71a8069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:52.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.871+0000 7f71b9ffb700 1 -- 172.21.15.110:0/1619930074 <== mon.2 v2:172.21.15.110:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5641+0+0 (secure 0 0 0) 0x7f71bc0861a0 con 0x7f71c40a4b20 2024-09-15T03:40:52.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.871+0000 7f71b9ffb700 1 --2- 172.21.15.110:0/1619930074 >> [v2:172.21.15.110:6816/22355,v1:172.21.15.110:6817/22355] conn(0x7f71a806ebd0 0x7f71a8070fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:52.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.871+0000 7f71b9ffb700 1 -- 172.21.15.110:0/1619930074 --> [v2:172.21.15.110:6816/22355,v1:172.21.15.110:6817/22355] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f71a8071690 con 0x7f71a806ebd0 2024-09-15T03:40:52.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.871+0000 7f71b9ffb700 1 -- 172.21.15.110:0/1619930074 <== mon.2 v2:172.21.15.110:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71bc0887e0 con 0x7f71c40a4b20 2024-09-15T03:40:52.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.871+0000 7f71ca538700 1 --2- 172.21.15.110:0/1619930074 >> [v2:172.21.15.110:6816/22355,v1:172.21.15.110:6817/22355] conn(0x7f71a806ebd0 0x7f71a8070fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:52.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.871+0000 7f71c9536700 1 --2- 172.21.15.110:0/1619930074 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f71a8067a80 0x7f71a8069f30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f71c40a4ef0 tx=0x7f71b4017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:52.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.871+0000 7f71ca538700 1 --2- 172.21.15.110:0/1619930074 >> [v2:172.21.15.110:6816/22355,v1:172.21.15.110:6817/22355] conn(0x7f71a806ebd0 0x7f71a8070fe0 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:52.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.871+0000 7f71b9ffb700 1 -- 172.21.15.110:0/1619930074 <== osd.3 v2:172.21.15.110:6816/22355 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f71a8071690 con 0x7f71a806ebd0 2024-09-15T03:40:52.892 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.887+0000 7f71cad39700 1 -- 172.21.15.110:0/1619930074 --> [v2:172.21.15.110:6816/22355,v1:172.21.15.110:6817/22355] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f71c414e1d0 con 0x7f71a806ebd0 2024-09-15T03:40:52.892 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.887+0000 7f71b9ffb700 1 -- 172.21.15.110:0/1619930074 <== osd.3 v2:172.21.15.110:6816/22355 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f71c414e1d0 con 0x7f71a806ebd0 2024-09-15T03:40:52.893 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-15T03:40:52.893 INFO:teuthology.orchestra.run.smithi110.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T03:40:52.893 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-15T03:40:52.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.887+0000 7f71cad39700 1 -- 172.21.15.110:0/1619930074 >> [v2:172.21.15.110:6816/22355,v1:172.21.15.110:6817/22355] conn(0x7f71a806ebd0 msgr2=0x7f71a8070fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:52.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.887+0000 7f71cad39700 1 --2- 172.21.15.110:0/1619930074 >> [v2:172.21.15.110:6816/22355,v1:172.21.15.110:6817/22355] conn(0x7f71a806ebd0 0x7f71a8070fe0 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:52.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.887+0000 7f71cad39700 1 -- 172.21.15.110:0/1619930074 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f71a8067a80 msgr2=0x7f71a8069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:52.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.887+0000 7f71cad39700 1 --2- 172.21.15.110:0/1619930074 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f71a8067a80 0x7f71a8069f30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f71c40a4ef0 tx=0x7f71b4017000 comp rx=0 tx=0).stop 2024-09-15T03:40:52.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.887+0000 7f71cad39700 1 -- 172.21.15.110:0/1619930074 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f71c40a4b20 msgr2=0x7f71c4134f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:52.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.887+0000 7f71cad39700 1 --2- 172.21.15.110:0/1619930074 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f71c40a4b20 0x7f71c4134f10 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f71bc009ef0 tx=0x7f71bc00b040 comp rx=0 tx=0).stop 2024-09-15T03:40:52.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.887+0000 7f71ca538700 1 -- 172.21.15.110:0/1619930074 reap_dead start 2024-09-15T03:40:52.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.887+0000 7f71cad39700 1 -- 172.21.15.110:0/1619930074 shutdown_connections 2024-09-15T03:40:52.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.887+0000 7f71cad39700 1 -- 172.21.15.110:0/1619930074 >> 172.21.15.110:0/1619930074 conn(0x7f71c40b1500 msgr2=0x7f71c406e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:40:52.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.891+0000 7f71cad39700 1 -- 172.21.15.110:0/1619930074 shutdown_connections 2024-09-15T03:40:52.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:52.891+0000 7f71cad39700 1 -- 172.21.15.110:0/1619930074 wait complete. 2024-09-15T03:40:52.905 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-15T03:40:53.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.015+0000 7f433759e700 1 Processor -- start 2024-09-15T03:40:53.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.015+0000 7f433759e700 1 -- start start 2024-09-15T03:40:53.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.015+0000 7f433759e700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f43380b0e00 0x7f43380a5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:53.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.015+0000 7f433759e700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f433813f7d0 0x7f433813b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:53.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.015+0000 7f433759e700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f433813f010 con 0x7f43380b0a30 2024-09-15T03:40:53.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.015+0000 7f433759e700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43380a6830 con 0x7f43380b0e00 2024-09-15T03:40:53.020 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.015+0000 7f433759e700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f433813f2b0 con 0x7f433813f7d0 2024-09-15T03:40:53.020 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.015+0000 7f4336d9d700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f433813f7d0 0x7f433813b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:53.020 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.015+0000 7f4335d9b700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f43380b0e00 0x7f43380a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:53.020 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.015+0000 7f4336d9d700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f433813f7d0 0x7f433813b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3301/0 says I am v2:172.21.15.110:43186/0 (socket says 172.21.15.110:43186) 2024-09-15T03:40:53.020 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.015+0000 7f4336d9d700 1 -- 172.21.15.110:0/610781874 learned_addr learned my addr 172.21.15.110:0/610781874 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:40:53.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.015+0000 7f433559a700 1 -- 172.21.15.110:0/610781874 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2267975591 0 0) 0x7f433813f010 con 0x7f43380b0a30 2024-09-15T03:40:53.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.015+0000 7f433559a700 1 -- 172.21.15.110:0/610781874 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4318003570 con 0x7f43380b0a30 2024-09-15T03:40:53.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.015+0000 7f4336d9d700 1 -- 172.21.15.110:0/610781874 >> v2:172.21.15.177:3300/0 conn(0x7f43380b0e00 msgr2=0x7f43380a5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:40:53.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.015+0000 7f4336d9d700 1 --2- 172.21.15.110:0/610781874 >> v2:172.21.15.177:3300/0 conn(0x7f43380b0e00 0x7f43380a5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:53.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.015+0000 7f4336d9d700 1 -- 172.21.15.110:0/610781874 >> v1:172.21.15.177:6789/0 conn(0x7f43380b0a30 legacy=0x7f433813bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:53.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.015+0000 7f4336d9d700 1 -- 172.21.15.110:0/610781874 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43380acb70 con 0x7f433813f7d0 2024-09-15T03:40:53.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.015+0000 7f4335d9b700 1 --2- 172.21.15.110:0/610781874 >> v2:172.21.15.177:3300/0 conn(0x7f43380b0e00 0x7f43380a5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T03:40:53.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.015+0000 7f4336d9d700 1 --2- 172.21.15.110:0/610781874 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f433813f7d0 0x7f433813b140 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f43200099b0 tx=0x7f4320009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b4150664d060ebd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:53.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.015+0000 7f433559a700 1 -- 172.21.15.110:0/610781874 <== mon.2 v2:172.21.15.110:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f432000b680 con 0x7f433813f7d0 2024-09-15T03:40:53.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.015+0000 7f433559a700 1 -- 172.21.15.110:0/610781874 <== mon.2 v2:172.21.15.110:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f432000b7e0 con 0x7f433813f7d0 2024-09-15T03:40:53.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.015+0000 7f433559a700 1 -- 172.21.15.110:0/610781874 <== mon.2 v2:172.21.15.110:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f432000bb60 con 0x7f433813f7d0 2024-09-15T03:40:53.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.019+0000 7f433759e700 1 -- 172.21.15.110:0/610781874 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f433813f7d0 msgr2=0x7f433813b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:53.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.019+0000 7f433759e700 1 --2- 172.21.15.110:0/610781874 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f433813f7d0 0x7f433813b140 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f43200099b0 tx=0x7f4320009250 comp rx=0 tx=0).stop 2024-09-15T03:40:53.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.019+0000 7f433759e700 1 -- 172.21.15.110:0/610781874 shutdown_connections 2024-09-15T03:40:53.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.019+0000 7f433759e700 1 --2- 172.21.15.110:0/610781874 >> v2:172.21.15.177:3300/0 conn(0x7f43380b0e00 0x7f43380a5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:53.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.019+0000 7f433759e700 1 --2- 172.21.15.110:0/610781874 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f433813f7d0 0x7f433813b140 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:53.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.019+0000 7f433759e700 1 -- 172.21.15.110:0/610781874 >> 172.21.15.110:0/610781874 conn(0x7f43380680f0 msgr2=0x7f43380b0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:40:53.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.019+0000 7f433759e700 1 -- 172.21.15.110:0/610781874 shutdown_connections 2024-09-15T03:40:53.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.019+0000 7f433759e700 1 -- 172.21.15.110:0/610781874 wait complete. 2024-09-15T03:40:53.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.019+0000 7f433759e700 1 Processor -- start 2024-09-15T03:40:53.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.019+0000 7f433759e700 1 -- start start 2024-09-15T03:40:53.026 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.023+0000 7f433759e700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f43380b0a30 0x7f4338150600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:53.026 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.023+0000 7f433759e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43380b0e00 0x7f4338141500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:53.026 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.023+0000 7f433759e700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f433813f7d0 0x7f433814d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:53.026 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.023+0000 7f433759e700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4338143470 con 0x7f433813f7d0 2024-09-15T03:40:53.026 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.023+0000 7f433759e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f433814d3f0 con 0x7f43380b0e00 2024-09-15T03:40:53.026 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.023+0000 7f433759e700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4338141a40 con 0x7f43380b0a30 2024-09-15T03:40:53.026 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.023+0000 7f4336d9d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f433813f7d0 0x7f433814d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:53.026 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.023+0000 7f433659c700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f43380b0a30 0x7f4338150600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:53.026 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.023+0000 7f4336d9d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f433813f7d0 0x7f433814d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51198/0 (socket says 172.21.15.110:51198) 2024-09-15T03:40:53.026 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.023+0000 7f4336d9d700 1 -- 172.21.15.110:0/2445131253 learned_addr learned my addr 172.21.15.110:0/2445131253 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:40:53.026 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.023+0000 7f4335d9b700 1 --2- 172.21.15.110:0/2445131253 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43380b0e00 0x7f4338141500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:53.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.023+0000 7f4336d9d700 1 -- 172.21.15.110:0/2445131253 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f43380b0a30 msgr2=0x7f4338150600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:53.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.023+0000 7f4336d9d700 1 --2- 172.21.15.110:0/2445131253 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f43380b0a30 0x7f4338150600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:53.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.023+0000 7f4336d9d700 1 -- 172.21.15.110:0/2445131253 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43380b0e00 msgr2=0x7f4338141500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:53.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.023+0000 7f4336d9d700 1 --2- 172.21.15.110:0/2445131253 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43380b0e00 0x7f4338141500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:53.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.023+0000 7f4336d9d700 1 -- 172.21.15.110:0/2445131253 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4328009420 con 0x7f433813f7d0 2024-09-15T03:40:53.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.023+0000 7f4336d9d700 1 --2- 172.21.15.110:0/2445131253 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f433813f7d0 0x7f433814d880 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f433813f3f0 tx=0x7f4320017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:53.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.023+0000 7f432e7fc700 1 -- 172.21.15.110:0/2445131253 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f432000bb60 con 0x7f433813f7d0 2024-09-15T03:40:53.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.023+0000 7f432e7fc700 1 -- 172.21.15.110:0/2445131253 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4320027450 con 0x7f433813f7d0 2024-09-15T03:40:53.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.023+0000 7f433759e700 1 -- 172.21.15.110:0/2445131253 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43200094f0 con 0x7f433813f7d0 2024-09-15T03:40:53.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.023+0000 7f432e7fc700 1 -- 172.21.15.110:0/2445131253 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4320027810 con 0x7f433813f7d0 2024-09-15T03:40:53.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.023+0000 7f433759e700 1 -- 172.21.15.110:0/2445131253 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4338151140 con 0x7f433813f7d0 2024-09-15T03:40:53.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.023+0000 7f433759e700 1 -- 172.21.15.110:0/2445131253 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f433814d580 con 0x7f433813f7d0 2024-09-15T03:40:53.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.027+0000 7f432e7fc700 1 -- 172.21.15.110:0/2445131253 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) v1 ==== 85847+0+0 (secure 0 0 0) 0x7f4320027b80 con 0x7f433813f7d0 2024-09-15T03:40:53.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.027+0000 7f432e7fc700 1 --2- 172.21.15.110:0/2445131253 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f431c067a80 0x7f431c069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:53.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.027+0000 7f432e7fc700 1 -- 172.21.15.110:0/2445131253 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5641+0+0 (secure 0 0 0) 0x7f4320087580 con 0x7f433813f7d0 2024-09-15T03:40:53.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.027+0000 7f433659c700 1 --2- 172.21.15.110:0/2445131253 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f431c067a80 0x7f431c069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:53.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.027+0000 7f433659c700 1 --2- 172.21.15.110:0/2445131253 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f431c067a80 0x7f431c069f30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4330000f80 tx=0x7f4330002720 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:53.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.035+0000 7f432e7fc700 1 -- 172.21.15.110:0/2445131253 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f4320054780 con 0x7f433813f7d0 2024-09-15T03:40:53.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.199+0000 7f433759e700 1 -- 172.21.15.110:0/2445131253 --> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f43380721b0 con 0x7f431c067a80 2024-09-15T03:40:53.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.199+0000 7f432e7fc700 1 -- 172.21.15.110:0/2445131253 <== mgr.4104 v2:172.21.15.110:6832/22190 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f43380721b0 con 0x7f431c067a80 2024-09-15T03:40:53.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.199+0000 7f433759e700 1 -- 172.21.15.110:0/2445131253 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f431c067a80 msgr2=0x7f431c069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:53.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.199+0000 7f433759e700 1 --2- 172.21.15.110:0/2445131253 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f431c067a80 0x7f431c069f30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4330000f80 tx=0x7f4330002720 comp rx=0 tx=0).stop 2024-09-15T03:40:53.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.199+0000 7f433759e700 1 -- 172.21.15.110:0/2445131253 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f433813f7d0 msgr2=0x7f433814d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:53.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.199+0000 7f433759e700 1 --2- 172.21.15.110:0/2445131253 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f433813f7d0 0x7f433814d880 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f433813f3f0 tx=0x7f4320017000 comp rx=0 tx=0).stop 2024-09-15T03:40:53.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.199+0000 7f433759e700 1 -- 172.21.15.110:0/2445131253 shutdown_connections 2024-09-15T03:40:53.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.199+0000 7f433759e700 1 --2- 172.21.15.110:0/2445131253 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f431c067a80 0x7f431c069f30 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:53.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.199+0000 7f433759e700 1 --2- 172.21.15.110:0/2445131253 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f433813f7d0 0x7f433814d880 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:53.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.199+0000 7f433759e700 1 --2- 172.21.15.110:0/2445131253 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43380b0e00 0x7f4338141500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:53.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.199+0000 7f433759e700 1 --2- 172.21.15.110:0/2445131253 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f43380b0a30 0x7f4338150600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:53.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.199+0000 7f433759e700 1 -- 172.21.15.110:0/2445131253 >> 172.21.15.110:0/2445131253 conn(0x7f43380680f0 msgr2=0x7f43380abcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:40:53.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.199+0000 7f433759e700 1 -- 172.21.15.110:0/2445131253 shutdown_connections 2024-09-15T03:40:53.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.199+0000 7f433759e700 1 -- 172.21.15.110:0/2445131253 wait complete. 2024-09-15T03:40:53.206 INFO:teuthology.orchestra.run.smithi110.stderr:instructed osd(s) 3 to deep-scrub 2024-09-15T03:40:53.217 INFO:tasks.ceph:Scrubbing osd.4 2024-09-15T03:40:53.217 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-15T03:40:53.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.319+0000 7f16af59e700 1 Processor -- start 2024-09-15T03:40:53.325 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.323+0000 7f16af59e700 1 -- start start 2024-09-15T03:40:53.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.323+0000 7f16af59e700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f16b0141380 0x7f16b006cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:53.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.323+0000 7f16af59e700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f16b0142060 0x7f16b00b09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:53.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.323+0000 7f16af59e700 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16b006f730 con 0x7f16b0141750 2024-09-15T03:40:53.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.323+0000 7f16af59e700 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16b00b1610 con 0x7f16b0141380 2024-09-15T03:40:53.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.323+0000 7f16af59e700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16b00b1750 con 0x7f16b0142060 2024-09-15T03:40:53.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.323+0000 7f16ae59c700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f16b0141380 0x7f16b006cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:53.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.323+0000 7f16ae59c700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f16b0141380 0x7f16b006cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51202/0 (socket says 172.21.15.110:51202) 2024-09-15T03:40:53.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.323+0000 7f16ae59c700 1 -- 172.21.15.110:0/16040353 learned_addr learned my addr 172.21.15.110:0/16040353 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:40:53.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.323+0000 7f16add9b700 1 --2- 172.21.15.110:0/16040353 >> v2:172.21.15.177:3300/0 conn(0x7f16b0142060 0x7f16b00b09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:53.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.323+0000 7f16ae59c700 1 -- 172.21.15.110:0/16040353 >> v2:172.21.15.177:3300/0 conn(0x7f16b0142060 msgr2=0x7f16b00b09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:40:53.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.323+0000 7f16ae59c700 1 --2- 172.21.15.110:0/16040353 >> v2:172.21.15.177:3300/0 conn(0x7f16b0142060 0x7f16b00b09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:53.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.323+0000 7f16ae59c700 1 -- 172.21.15.110:0/16040353 >> v1:172.21.15.110:6789/0 conn(0x7f16b0141750 legacy=0x7f16b00b0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:53.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.323+0000 7f16ae59c700 1 -- 172.21.15.110:0/16040353 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16b01488d0 con 0x7f16b0141380 2024-09-15T03:40:53.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.323+0000 7f16ae59c700 1 --2- 172.21.15.110:0/16040353 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16b0141380 0x7f16b006cb50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f16b0142450 tx=0x7f16a0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88c222485fab5a48 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:53.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.323+0000 7f16ad59a700 1 -- 172.21.15.110:0/16040353 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a000e070 con 0x7f16b0141380 2024-09-15T03:40:53.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.323+0000 7f16ae59c700 1 -- 172.21.15.110:0/16040353 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16b0141380 msgr2=0x7f16b006cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-15T03:40:53.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.323+0000 7f16ae59c700 1 -- 172.21.15.110:0/16040353 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16b0141380 msgr2=0x7f16b006cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T03:40:53.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.323+0000 7f16ae59c700 1 --2- 172.21.15.110:0/16040353 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16b0141380 0x7f16b006cb50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f16b0142450 tx=0x7f16a0009840 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T03:40:53.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.323+0000 7f16ae59c700 1 --2- 172.21.15.110:0/16040353 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16b0141380 0x7f16b006cb50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f16b0142450 tx=0x7f16a0009840 comp rx=0 tx=0).stop 2024-09-15T03:40:53.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.323+0000 7f16ad59a700 1 -- 172.21.15.110:0/16040353 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16b0141380 msgr2=0x7f16b006cb50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:40:53.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.323+0000 7f16ad59a700 1 --2- 172.21.15.110:0/16040353 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16b0141380 0x7f16b006cb50 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:53.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.323+0000 7f16ad59a700 1 --2- 172.21.15.110:0/16040353 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f169c003750 0x7f169c005b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:53.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.323+0000 7f16ad59a700 1 --2- 172.21.15.110:0/16040353 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f169c0063b0 0x7f169c008830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:53.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.323+0000 7f16ad59a700 1 --2- 172.21.15.110:0/16040353 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f169c009110 0x7f169c00b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:53.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.323+0000 7f16ad59a700 1 -- 172.21.15.110:0/16040353 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f169c00bf00 con 0x7f169c0063b0 2024-09-15T03:40:53.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.323+0000 7f16ad59a700 1 -- 172.21.15.110:0/16040353 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f169c00c070 con 0x7f169c003750 2024-09-15T03:40:53.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.323+0000 7f16ad59a700 1 -- 172.21.15.110:0/16040353 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f169c00c230 con 0x7f169c009110 2024-09-15T03:40:53.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.323+0000 7f16add9b700 1 --2- 172.21.15.110:0/16040353 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f169c0063b0 0x7f169c008830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:53.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.323+0000 7f16aed9d700 1 --2- 172.21.15.110:0/16040353 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f169c009110 0x7f169c00b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:53.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.323+0000 7f16ae59c700 1 --2- 172.21.15.110:0/16040353 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f169c003750 0x7f169c005b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:53.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.323+0000 7f16add9b700 1 -- 172.21.15.110:0/16040353 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f169c009110 msgr2=0x7f169c00b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:40:53.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.323+0000 7f16add9b700 1 --2- 172.21.15.110:0/16040353 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f169c009110 0x7f169c00b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:53.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.323+0000 7f16add9b700 1 -- 172.21.15.110:0/16040353 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f169c003750 msgr2=0x7f169c005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:40:53.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.323+0000 7f16add9b700 1 --2- 172.21.15.110:0/16040353 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f169c003750 0x7f169c005b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:53.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.323+0000 7f16add9b700 1 -- 172.21.15.110:0/16040353 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=2+}) v3 -- 0x7f169c00c4b0 con 0x7f169c0063b0 2024-09-15T03:40:53.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.323+0000 7f16ae59c700 1 --2- 172.21.15.110:0/16040353 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f169c003750 0x7f169c005b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T03:40:53.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.323+0000 7f16aed9d700 1 -- 172.21.15.110:0/16040353 reap_dead start 2024-09-15T03:40:53.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.323+0000 7f16add9b700 1 --2- 172.21.15.110:0/16040353 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f169c0063b0 0x7f169c008830 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1698009c00 tx=0x7f16980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dad284c1facd1706 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:53.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.323+0000 7f16ad59a700 1 -- 172.21.15.110:0/16040353 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169800e710 con 0x7f169c0063b0 2024-09-15T03:40:53.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.323+0000 7f16ad59a700 1 -- 172.21.15.110:0/16040353 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f169800e870 con 0x7f169c0063b0 2024-09-15T03:40:53.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.327+0000 7f16af59e700 1 -- 172.21.15.110:0/16040353 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f169c0063b0 msgr2=0x7f169c008830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:53.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.327+0000 7f16af59e700 1 --2- 172.21.15.110:0/16040353 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f169c0063b0 0x7f169c008830 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1698009c00 tx=0x7f16980091a0 comp rx=0 tx=0).stop 2024-09-15T03:40:53.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.327+0000 7f16af59e700 1 -- 172.21.15.110:0/16040353 shutdown_connections 2024-09-15T03:40:53.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.327+0000 7f16af59e700 1 --2- 172.21.15.110:0/16040353 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f169c0063b0 0x7f169c008830 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:53.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.327+0000 7f16af59e700 1 --2- 172.21.15.110:0/16040353 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16b0141380 0x7f16b006cb50 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:53.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.327+0000 7f16af59e700 1 -- 172.21.15.110:0/16040353 >> 172.21.15.110:0/16040353 conn(0x7f16b0068de0 msgr2=0x7f16b00691e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:40:53.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.327+0000 7f16af59e700 1 -- 172.21.15.110:0/16040353 shutdown_connections 2024-09-15T03:40:53.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.327+0000 7f16af59e700 1 -- 172.21.15.110:0/16040353 wait complete. 2024-09-15T03:40:53.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.327+0000 7f16af59e700 1 Processor -- start 2024-09-15T03:40:53.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.327+0000 7f16af59e700 1 -- start start 2024-09-15T03:40:53.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.327+0000 7f16af59e700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f169c0063b0 0x7f16b0137d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:53.336 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.327+0000 7f16af59e700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f16b0141380 0x7f16b013e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:53.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.327+0000 7f16af59e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f16b01484b0 0x7f16b013c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:53.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.327+0000 7f16af59e700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b01363f0 con 0x7f169c0063b0 2024-09-15T03:40:53.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.327+0000 7f16af59e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b013ecc0 con 0x7f16b01484b0 2024-09-15T03:40:53.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.327+0000 7f16af59e700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16b013e7d0 con 0x7f16b0141380 2024-09-15T03:40:53.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.327+0000 7f16ae59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f169c0063b0 0x7f16b0137d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:53.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.327+0000 7f16add9b700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f16b0141380 0x7f16b013e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:53.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.327+0000 7f16add9b700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f16b0141380 0x7f16b013e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3301/0 says I am v2:172.21.15.110:43226/0 (socket says 172.21.15.110:43226) 2024-09-15T03:40:53.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.327+0000 7f16ae59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f169c0063b0 0x7f16b0137d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51222/0 (socket says 172.21.15.110:51222) 2024-09-15T03:40:53.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.327+0000 7f16add9b700 1 -- 172.21.15.110:0/1112852998 learned_addr learned my addr 172.21.15.110:0/1112852998 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:40:53.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.327+0000 7f16aed9d700 1 --2- 172.21.15.110:0/1112852998 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f16b01484b0 0x7f16b013c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:53.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.327+0000 7f16add9b700 1 -- 172.21.15.110:0/1112852998 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f16b01484b0 msgr2=0x7f16b013c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:53.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.327+0000 7f16add9b700 1 --2- 172.21.15.110:0/1112852998 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f16b01484b0 0x7f16b013c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:53.338 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.327+0000 7f16add9b700 1 -- 172.21.15.110:0/1112852998 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f169c0063b0 msgr2=0x7f16b0137d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:53.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.327+0000 7f16add9b700 1 --2- 172.21.15.110:0/1112852998 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f169c0063b0 0x7f16b0137d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:53.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.327+0000 7f16add9b700 1 -- 172.21.15.110:0/1112852998 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16b013e990 con 0x7f16b0141380 2024-09-15T03:40:53.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.331+0000 7f16add9b700 1 --2- 172.21.15.110:0/1112852998 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f16b0141380 0x7f16b013e290 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f169800e4b0 tx=0x7f169801a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:53.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.331+0000 7f16a67fc700 1 -- 172.21.15.110:0/1112852998 <== mon.2 v2:172.21.15.110:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1698022070 con 0x7f16b0141380 2024-09-15T03:40:53.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.331+0000 7f16a67fc700 1 -- 172.21.15.110:0/1112852998 <== mon.2 v2:172.21.15.110:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1698013c60 con 0x7f16b0141380 2024-09-15T03:40:53.339 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.331+0000 7f16a67fc700 1 -- 172.21.15.110:0/1112852998 <== mon.2 v2:172.21.15.110:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169801c680 con 0x7f16b0141380 2024-09-15T03:40:53.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.331+0000 7f16af59e700 1 -- 172.21.15.110:0/1112852998 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16b0135180 con 0x7f16b0141380 2024-09-15T03:40:53.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.331+0000 7f16af59e700 1 -- 172.21.15.110:0/1112852998 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16b013c810 con 0x7f16b0141380 2024-09-15T03:40:53.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.331+0000 7f16af59e700 1 -- 172.21.15.110:0/1112852998 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f16b013ca90 con 0x7f16b0141380 2024-09-15T03:40:53.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.331+0000 7f16a67fc700 1 -- 172.21.15.110:0/1112852998 <== mon.2 v2:172.21.15.110:3301/0 4 ==== mgrmap(e 5) v1 ==== 85847+0+0 (secure 0 0 0) 0x7f169801cab0 con 0x7f16b0141380 2024-09-15T03:40:53.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.331+0000 7f16a67fc700 1 --2- 172.21.15.110:0/1112852998 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f1690067a80 0x7f1690069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:53.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.331+0000 7f16a67fc700 1 -- 172.21.15.110:0/1112852998 <== mon.2 v2:172.21.15.110:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5641+0+0 (secure 0 0 0) 0x7f1698087900 con 0x7f16b0141380 2024-09-15T03:40:53.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.331+0000 7f16ae59c700 1 --2- 172.21.15.110:0/1112852998 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f1690067a80 0x7f1690069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:53.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.331+0000 7f16a67fc700 1 --2- 172.21.15.110:0/1112852998 >> [v2:172.21.15.177:6822/22425,v1:172.21.15.177:6824/22425] conn(0x7f169006ebd0 0x7f1690070fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:53.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.331+0000 7f16a67fc700 1 -- 172.21.15.110:0/1112852998 --> [v2:172.21.15.177:6822/22425,v1:172.21.15.177:6824/22425] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1690071690 con 0x7f169006ebd0 2024-09-15T03:40:53.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.331+0000 7f16a67fc700 1 -- 172.21.15.110:0/1112852998 <== mon.2 v2:172.21.15.110:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1698087cd0 con 0x7f16b0141380 2024-09-15T03:40:53.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.335+0000 7f16aed9d700 1 --2- 172.21.15.110:0/1112852998 >> [v2:172.21.15.177:6822/22425,v1:172.21.15.177:6824/22425] conn(0x7f169006ebd0 0x7f1690070fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:53.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.335+0000 7f16ae59c700 1 --2- 172.21.15.110:0/1112852998 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f1690067a80 0x7f1690069f30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f16a0009e70 tx=0x7f16a0018000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:53.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.335+0000 7f16aed9d700 1 --2- 172.21.15.110:0/1112852998 >> [v2:172.21.15.177:6822/22425,v1:172.21.15.177:6824/22425] conn(0x7f169006ebd0 0x7f1690070fe0 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:53.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.335+0000 7f16a67fc700 1 -- 172.21.15.110:0/1112852998 <== osd.4 v2:172.21.15.177:6822/22425 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f1690071690 con 0x7f169006ebd0 2024-09-15T03:40:53.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.355+0000 7f16af59e700 1 -- 172.21.15.110:0/1112852998 --> [v2:172.21.15.177:6822/22425,v1:172.21.15.177:6824/22425] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f16b00b2890 con 0x7f169006ebd0 2024-09-15T03:40:53.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.355+0000 7f16a67fc700 1 -- 172.21.15.110:0/1112852998 <== osd.4 v2:172.21.15.177:6822/22425 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f16b00b2890 con 0x7f169006ebd0 2024-09-15T03:40:53.359 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-15T03:40:53.360 INFO:teuthology.orchestra.run.smithi110.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T03:40:53.360 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-15T03:40:53.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.355+0000 7f16af59e700 1 -- 172.21.15.110:0/1112852998 >> [v2:172.21.15.177:6822/22425,v1:172.21.15.177:6824/22425] conn(0x7f169006ebd0 msgr2=0x7f1690070fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:53.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.355+0000 7f16af59e700 1 --2- 172.21.15.110:0/1112852998 >> [v2:172.21.15.177:6822/22425,v1:172.21.15.177:6824/22425] conn(0x7f169006ebd0 0x7f1690070fe0 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:53.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.355+0000 7f16af59e700 1 -- 172.21.15.110:0/1112852998 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f1690067a80 msgr2=0x7f1690069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:53.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.355+0000 7f16af59e700 1 --2- 172.21.15.110:0/1112852998 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f1690067a80 0x7f1690069f30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f16a0009e70 tx=0x7f16a0018000 comp rx=0 tx=0).stop 2024-09-15T03:40:53.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.355+0000 7f16af59e700 1 -- 172.21.15.110:0/1112852998 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f16b0141380 msgr2=0x7f16b013e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:53.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.355+0000 7f16af59e700 1 --2- 172.21.15.110:0/1112852998 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f16b0141380 0x7f16b013e290 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f169800e4b0 tx=0x7f169801a000 comp rx=0 tx=0).stop 2024-09-15T03:40:53.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.355+0000 7f16aed9d700 1 -- 172.21.15.110:0/1112852998 reap_dead start 2024-09-15T03:40:53.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.355+0000 7f16af59e700 1 -- 172.21.15.110:0/1112852998 shutdown_connections 2024-09-15T03:40:53.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.355+0000 7f16af59e700 1 -- 172.21.15.110:0/1112852998 >> 172.21.15.110:0/1112852998 conn(0x7f16b0068de0 msgr2=0x7f16b013f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:40:53.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.355+0000 7f16af59e700 1 -- 172.21.15.110:0/1112852998 shutdown_connections 2024-09-15T03:40:53.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.355+0000 7f16af59e700 1 -- 172.21.15.110:0/1112852998 wait complete. 2024-09-15T03:40:53.373 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-15T03:40:53.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.491+0000 7f890aee0700 1 Processor -- start 2024-09-15T03:40:53.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.491+0000 7f890aee0700 1 -- start start 2024-09-15T03:40:53.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.491+0000 7f890aee0700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f89040b1a40 0x7f8904146070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:53.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.491+0000 7f890aee0700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f8904145630 0x7f89040ad4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:53.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.491+0000 7f890aee0700 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8904145be0 con 0x7f89040b1e10 2024-09-15T03:40:53.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.491+0000 7f890aee0700 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89041465b0 con 0x7f89040b1a40 2024-09-15T03:40:53.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.491+0000 7f890aee0700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8904141f30 con 0x7f8904145630 2024-09-15T03:40:53.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.491+0000 7f8909ede700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f89040b1a40 0x7f8904146070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:53.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.491+0000 7f8909ede700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f89040b1a40 0x7f8904146070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51236/0 (socket says 172.21.15.110:51236) 2024-09-15T03:40:53.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.491+0000 7f8909ede700 1 -- 172.21.15.110:0/3801813870 learned_addr learned my addr 172.21.15.110:0/3801813870 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:40:53.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.491+0000 7f890a6df700 1 --2- 172.21.15.110:0/3801813870 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f8904145630 0x7f89040ad4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:53.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.491+0000 7f8909ede700 1 -- 172.21.15.110:0/3801813870 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f8904145630 msgr2=0x7f89040ad4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:40:53.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.491+0000 7f8909ede700 1 --2- 172.21.15.110:0/3801813870 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f8904145630 0x7f89040ad4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:53.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.491+0000 7f8909ede700 1 -- 172.21.15.110:0/3801813870 >> v1:172.21.15.110:6789/0 conn(0x7f89040b1e10 legacy=0x7f890406e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:53.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.491+0000 7f8909ede700 1 -- 172.21.15.110:0/3801813870 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89040ada20 con 0x7f89040b1a40 2024-09-15T03:40:53.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.491+0000 7f890a6df700 1 --2- 172.21.15.110:0/3801813870 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f8904145630 0x7f89040ad4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T03:40:53.498 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.491+0000 7f8909ede700 1 --2- 172.21.15.110:0/3801813870 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f89040b1a40 0x7f8904146070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f89000099b0 tx=0x7f8900009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73bf272dc64f9c45 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:53.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.491+0000 7f8908edc700 1 -- 172.21.15.110:0/3801813870 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8900009ea0 con 0x7f89040b1a40 2024-09-15T03:40:53.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.491+0000 7f8908edc700 1 -- 172.21.15.110:0/3801813870 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f890000b860 con 0x7f89040b1a40 2024-09-15T03:40:53.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.495+0000 7f8908edc700 1 -- 172.21.15.110:0/3801813870 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f890000bc00 con 0x7f89040b1a40 2024-09-15T03:40:53.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.495+0000 7f890aee0700 1 -- 172.21.15.110:0/3801813870 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f89040b1a40 msgr2=0x7f8904146070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:53.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.495+0000 7f890aee0700 1 --2- 172.21.15.110:0/3801813870 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f89040b1a40 0x7f8904146070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f89000099b0 tx=0x7f8900009250 comp rx=0 tx=0).stop 2024-09-15T03:40:53.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.495+0000 7f890aee0700 1 -- 172.21.15.110:0/3801813870 shutdown_connections 2024-09-15T03:40:53.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.495+0000 7f890aee0700 1 --2- 172.21.15.110:0/3801813870 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f8904145630 0x7f89040ad4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:53.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.495+0000 7f890aee0700 1 --2- 172.21.15.110:0/3801813870 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f89040b1a40 0x7f8904146070 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:53.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.495+0000 7f890aee0700 1 -- 172.21.15.110:0/3801813870 >> 172.21.15.110:0/3801813870 conn(0x7f89040680f0 msgr2=0x7f89040b1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:40:53.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.495+0000 7f890aee0700 1 -- 172.21.15.110:0/3801813870 shutdown_connections 2024-09-15T03:40:53.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.495+0000 7f890aee0700 1 -- 172.21.15.110:0/3801813870 wait complete. 2024-09-15T03:40:53.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.495+0000 7f890aee0700 1 Processor -- start 2024-09-15T03:40:53.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.495+0000 7f890aee0700 1 -- start start 2024-09-15T03:40:53.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.495+0000 7f890aee0700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89040b1a40 0x7f8904154d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:53.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.495+0000 7f890aee0700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f89040b1e10 0x7f89040b03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:53.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.495+0000 7f890aee0700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f8904145630 0x7f89040b08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:53.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.495+0000 7f890aee0700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89040aff10 con 0x7f89040b1e10 2024-09-15T03:40:53.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.495+0000 7f890aee0700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f890413ceb0 con 0x7f89040b1a40 2024-09-15T03:40:53.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.495+0000 7f890aee0700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89040afb10 con 0x7f8904145630 2024-09-15T03:40:53.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.495+0000 7f890a6df700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f8904145630 0x7f89040b08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:53.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.495+0000 7f890a6df700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f8904145630 0x7f89040b08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3301/0 says I am v2:172.21.15.110:43242/0 (socket says 172.21.15.110:43242) 2024-09-15T03:40:53.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.499+0000 7f890a6df700 1 -- 172.21.15.110:0/1932907516 learned_addr learned my addr 172.21.15.110:0/1932907516 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:40:53.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.499+0000 7f89096dd700 1 --2- 172.21.15.110:0/1932907516 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f89040b1e10 0x7f89040b03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:53.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.499+0000 7f890a6df700 1 -- 172.21.15.110:0/1932907516 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89040b1a40 msgr2=0x7f8904154d80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-15T03:40:53.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.499+0000 7f890a6df700 1 --2- 172.21.15.110:0/1932907516 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89040b1a40 0x7f8904154d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:53.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.499+0000 7f890a6df700 1 -- 172.21.15.110:0/1932907516 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f89040b1e10 msgr2=0x7f89040b03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:53.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.499+0000 7f890a6df700 1 --2- 172.21.15.110:0/1932907516 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f89040b1e10 0x7f89040b03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:53.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.499+0000 7f890a6df700 1 -- 172.21.15.110:0/1932907516 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88fc009a20 con 0x7f8904145630 2024-09-15T03:40:53.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.499+0000 7f890a6df700 1 --2- 172.21.15.110:0/1932907516 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f8904145630 0x7f89040b08e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f88fc009f50 tx=0x7f88fc00b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:53.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.499+0000 7f88f9ffb700 1 -- 172.21.15.110:0/1932907516 <== mon.2 v2:172.21.15.110:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88fc00cc00 con 0x7f8904145630 2024-09-15T03:40:53.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.499+0000 7f88f9ffb700 1 -- 172.21.15.110:0/1932907516 <== mon.2 v2:172.21.15.110:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88fc00f040 con 0x7f8904145630 2024-09-15T03:40:53.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.499+0000 7f88f9ffb700 1 -- 172.21.15.110:0/1932907516 <== mon.2 v2:172.21.15.110:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88fc014480 con 0x7f8904145630 2024-09-15T03:40:53.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.499+0000 7f890aee0700 1 -- 172.21.15.110:0/1932907516 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89000094f0 con 0x7f8904145630 2024-09-15T03:40:53.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.499+0000 7f890aee0700 1 -- 172.21.15.110:0/1932907516 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8904155830 con 0x7f8904145630 2024-09-15T03:40:53.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.499+0000 7f890aee0700 1 -- 172.21.15.110:0/1932907516 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89040b00c0 con 0x7f8904145630 2024-09-15T03:40:53.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.499+0000 7f88f9ffb700 1 -- 172.21.15.110:0/1932907516 <== mon.2 v2:172.21.15.110:3301/0 4 ==== mgrmap(e 5) v1 ==== 85847+0+0 (secure 0 0 0) 0x7f88fc003020 con 0x7f8904145630 2024-09-15T03:40:53.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.503+0000 7f88f9ffb700 1 --2- 172.21.15.110:0/1932907516 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f88e8067ab0 0x7f88e8069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:53.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.503+0000 7f88f9ffb700 1 -- 172.21.15.110:0/1932907516 <== mon.2 v2:172.21.15.110:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5641+0+0 (secure 0 0 0) 0x7f88fc088030 con 0x7f8904145630 2024-09-15T03:40:53.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.503+0000 7f8909ede700 1 --2- 172.21.15.110:0/1932907516 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f88e8067ab0 0x7f88e8069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:53.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.503+0000 7f8909ede700 1 --2- 172.21.15.110:0/1932907516 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f88e8067ab0 0x7f88e8069f60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f89000099b0 tx=0x7f8900017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:53.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.507+0000 7f88f9ffb700 1 -- 172.21.15.110:0/1932907516 <== mon.2 v2:172.21.15.110:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f89040b00c0 con 0x7f8904145630 2024-09-15T03:40:53.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.703+0000 7f890aee0700 1 -- 172.21.15.110:0/1932907516 --> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f89040b00c0 con 0x7f88e8067ab0 2024-09-15T03:40:53.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.703+0000 7f88f9ffb700 1 -- 172.21.15.110:0/1932907516 <== mgr.4104 v2:172.21.15.110:6832/22190 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f89040b00c0 con 0x7f88e8067ab0 2024-09-15T03:40:53.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.707+0000 7f890aee0700 1 -- 172.21.15.110:0/1932907516 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f88e8067ab0 msgr2=0x7f88e8069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:53.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.707+0000 7f890aee0700 1 --2- 172.21.15.110:0/1932907516 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f88e8067ab0 0x7f88e8069f60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f89000099b0 tx=0x7f8900017000 comp rx=0 tx=0).stop 2024-09-15T03:40:53.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.707+0000 7f890aee0700 1 -- 172.21.15.110:0/1932907516 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f8904145630 msgr2=0x7f89040b08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:53.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.707+0000 7f890aee0700 1 --2- 172.21.15.110:0/1932907516 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f8904145630 0x7f89040b08e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f88fc009f50 tx=0x7f88fc00b040 comp rx=0 tx=0).stop 2024-09-15T03:40:53.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.707+0000 7f890aee0700 1 -- 172.21.15.110:0/1932907516 shutdown_connections 2024-09-15T03:40:53.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.707+0000 7f890aee0700 1 --2- 172.21.15.110:0/1932907516 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f88e8067ab0 0x7f88e8069f60 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:53.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.707+0000 7f890aee0700 1 --2- 172.21.15.110:0/1932907516 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f8904145630 0x7f89040b08e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:53.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.707+0000 7f890aee0700 1 --2- 172.21.15.110:0/1932907516 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f89040b1e10 0x7f89040b03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:53.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.707+0000 7f890aee0700 1 --2- 172.21.15.110:0/1932907516 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89040b1a40 0x7f8904154d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:53.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.707+0000 7f890aee0700 1 -- 172.21.15.110:0/1932907516 >> 172.21.15.110:0/1932907516 conn(0x7f89040680f0 msgr2=0x7f89040addb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:40:53.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.707+0000 7f890aee0700 1 -- 172.21.15.110:0/1932907516 shutdown_connections 2024-09-15T03:40:53.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.707+0000 7f890aee0700 1 -- 172.21.15.110:0/1932907516 wait complete. 2024-09-15T03:40:53.712 INFO:teuthology.orchestra.run.smithi110.stderr:instructed osd(s) 4 to deep-scrub 2024-09-15T03:40:53.722 INFO:tasks.ceph:Scrubbing osd.5 2024-09-15T03:40:53.722 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-15T03:40:53.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.827+0000 7ff0c27be700 1 Processor -- start 2024-09-15T03:40:53.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.827+0000 7ff0c27be700 1 -- start start 2024-09-15T03:40:53.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.827+0000 7ff0c27be700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7ff0bc0933b0 0x7ff0bc093df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:53.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.827+0000 7ff0c27be700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ff0bc06dce0 0x7ff0bc13ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:53.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.827+0000 7ff0c27be700 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0bc093960 con 0x7ff0bc0b0af0 2024-09-15T03:40:53.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.827+0000 7ff0c27be700 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0bc094330 con 0x7ff0bc06dce0 2024-09-15T03:40:53.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.827+0000 7ff0c27be700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0bc0b0ec0 con 0x7ff0bc0933b0 2024-09-15T03:40:53.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.827+0000 7ff0c1fbd700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ff0bc06dce0 0x7ff0bc13ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:53.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.827+0000 7ff0c0fbb700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7ff0bc0933b0 0x7ff0bc093df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:53.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.827+0000 7ff0c0fbb700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7ff0bc0933b0 0x7ff0bc093df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3301/0 says I am v2:172.21.15.110:43248/0 (socket says 172.21.15.110:43248) 2024-09-15T03:40:53.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.827+0000 7ff0c1fbd700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7ff0bc06dce0 0x7ff0bc13ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51254/0 (socket says 172.21.15.110:51254) 2024-09-15T03:40:53.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.827+0000 7ff0c0fbb700 1 -- 172.21.15.110:0/4189095928 learned_addr learned my addr 172.21.15.110:0/4189095928 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:40:53.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.827+0000 7ff0c1fbd700 1 -- 172.21.15.110:0/4189095928 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7ff0bc0933b0 msgr2=0x7ff0bc093df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:40:53.834 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.827+0000 7ff0c1fbd700 1 --2- 172.21.15.110:0/4189095928 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7ff0bc0933b0 0x7ff0bc093df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:53.834 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.827+0000 7ff0c1fbd700 1 -- 172.21.15.110:0/4189095928 >> v1:172.21.15.110:6789/0 conn(0x7ff0bc0b0af0 legacy=0x7ff0bc06c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:53.834 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.827+0000 7ff0c1fbd700 1 -- 172.21.15.110:0/4189095928 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0bc06e160 con 0x7ff0bc06dce0 2024-09-15T03:40:53.834 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.827+0000 7ff0c0fbb700 1 --2- 172.21.15.110:0/4189095928 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7ff0bc0933b0 0x7ff0bc093df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T03:40:53.834 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.827+0000 7ff0c1fbd700 1 --2- 172.21.15.110:0/4189095928 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff0bc06dce0 0x7ff0bc13ad70 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4009e60 tx=0x7ff0b4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b18eff3efff5c888 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:53.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.831+0000 7ff0affff700 1 -- 172.21.15.110:0/4189095928 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b400e070 con 0x7ff0bc06dce0 2024-09-15T03:40:53.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.831+0000 7ff0affff700 1 -- 172.21.15.110:0/4189095928 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0b400bb60 con 0x7ff0bc06dce0 2024-09-15T03:40:53.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.831+0000 7ff0affff700 1 -- 172.21.15.110:0/4189095928 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b4012430 con 0x7ff0bc06dce0 2024-09-15T03:40:53.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.831+0000 7ff0c27be700 1 -- 172.21.15.110:0/4189095928 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff0bc06dce0 msgr2=0x7ff0bc13ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:53.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.831+0000 7ff0c27be700 1 --2- 172.21.15.110:0/4189095928 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff0bc06dce0 0x7ff0bc13ad70 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4009e60 tx=0x7ff0b4009780 comp rx=0 tx=0).stop 2024-09-15T03:40:53.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.831+0000 7ff0c27be700 1 -- 172.21.15.110:0/4189095928 shutdown_connections 2024-09-15T03:40:53.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.831+0000 7ff0c27be700 1 --2- 172.21.15.110:0/4189095928 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff0bc06dce0 0x7ff0bc13ad70 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:53.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.831+0000 7ff0c27be700 1 --2- 172.21.15.110:0/4189095928 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7ff0bc0933b0 0x7ff0bc093df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:53.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.831+0000 7ff0c27be700 1 -- 172.21.15.110:0/4189095928 >> 172.21.15.110:0/4189095928 conn(0x7ff0bc069120 msgr2=0x7ff0bc0694f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:40:53.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.831+0000 7ff0c27be700 1 -- 172.21.15.110:0/4189095928 shutdown_connections 2024-09-15T03:40:53.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.831+0000 7ff0c27be700 1 -- 172.21.15.110:0/4189095928 wait complete. 2024-09-15T03:40:53.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.831+0000 7ff0c27be700 1 Processor -- start 2024-09-15T03:40:53.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.831+0000 7ff0c27be700 1 -- start start 2024-09-15T03:40:53.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.835+0000 7ff0c27be700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7ff0bc06dce0 0x7ff0bc0a9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:53.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.835+0000 7ff0c27be700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff0bc0933b0 0x7ff0bc087190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:53.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.835+0000 7ff0c27be700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff0bc0b0af0 0x7ff0bc0abc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:53.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.835+0000 7ff0c27be700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0bc0876d0 con 0x7ff0bc0b0af0 2024-09-15T03:40:53.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.835+0000 7ff0c27be700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0bc0ac160 con 0x7ff0bc0933b0 2024-09-15T03:40:53.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.835+0000 7ff0c27be700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0bc088df0 con 0x7ff0bc06dce0 2024-09-15T03:40:53.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.835+0000 7ff0c17bc700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7ff0bc06dce0 0x7ff0bc0a9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:53.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.835+0000 7ff0c1fbd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff0bc0b0af0 0x7ff0bc0abc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:53.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.835+0000 7ff0c17bc700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7ff0bc06dce0 0x7ff0bc0a9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3301/0 says I am v2:172.21.15.110:43258/0 (socket says 172.21.15.110:43258) 2024-09-15T03:40:53.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.835+0000 7ff0c17bc700 1 -- 172.21.15.110:0/199940307 learned_addr learned my addr 172.21.15.110:0/199940307 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:40:53.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.835+0000 7ff0c0fbb700 1 --2- 172.21.15.110:0/199940307 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff0bc0933b0 0x7ff0bc087190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:53.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.835+0000 7ff0c1fbd700 1 -- 172.21.15.110:0/199940307 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7ff0bc06dce0 msgr2=0x7ff0bc0a9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:53.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.835+0000 7ff0c1fbd700 1 --2- 172.21.15.110:0/199940307 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7ff0bc06dce0 0x7ff0bc0a9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:53.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.835+0000 7ff0c1fbd700 1 -- 172.21.15.110:0/199940307 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff0bc0933b0 msgr2=0x7ff0bc087190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:53.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.835+0000 7ff0c1fbd700 1 --2- 172.21.15.110:0/199940307 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff0bc0933b0 0x7ff0bc087190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:53.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.835+0000 7ff0c1fbd700 1 -- 172.21.15.110:0/199940307 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0b800b790 con 0x7ff0bc0b0af0 2024-09-15T03:40:53.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.835+0000 7ff0c0fbb700 1 --2- 172.21.15.110:0/199940307 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff0bc0933b0 0x7ff0bc087190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T03:40:53.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.835+0000 7ff0c1fbd700 1 --2- 172.21.15.110:0/199940307 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff0bc0b0af0 0x7ff0bc0abc20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4016000 tx=0x7ff0b4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:53.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.835+0000 7ff0ad7fa700 1 -- 172.21.15.110:0/199940307 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b4027470 con 0x7ff0bc0b0af0 2024-09-15T03:40:53.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.835+0000 7ff0ad7fa700 1 -- 172.21.15.110:0/199940307 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0b40275f0 con 0x7ff0bc0b0af0 2024-09-15T03:40:53.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.835+0000 7ff0c27be700 1 -- 172.21.15.110:0/199940307 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0b4009a20 con 0x7ff0bc0b0af0 2024-09-15T03:40:53.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.835+0000 7ff0ad7fa700 1 -- 172.21.15.110:0/199940307 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b40279b0 con 0x7ff0bc0b0af0 2024-09-15T03:40:53.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.835+0000 7ff0c27be700 1 -- 172.21.15.110:0/199940307 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0bc14def0 con 0x7ff0bc0b0af0 2024-09-15T03:40:53.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.835+0000 7ff0c27be700 1 -- 172.21.15.110:0/199940307 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff0bc14d860 con 0x7ff0bc0b0af0 2024-09-15T03:40:53.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.839+0000 7ff0ad7fa700 1 -- 172.21.15.110:0/199940307 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) v1 ==== 85847+0+0 (secure 0 0 0) 0x7ff0b4024070 con 0x7ff0bc0b0af0 2024-09-15T03:40:53.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.839+0000 7ff0ad7fa700 1 --2- 172.21.15.110:0/199940307 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7ff0a0067a80 0x7ff0a0069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:53.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.839+0000 7ff0ad7fa700 1 -- 172.21.15.110:0/199940307 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5641+0+0 (secure 0 0 0) 0x7ff0b40886b0 con 0x7ff0bc0b0af0 2024-09-15T03:40:53.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.839+0000 7ff0c17bc700 1 --2- 172.21.15.110:0/199940307 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7ff0a0067a80 0x7ff0a0069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:53.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.839+0000 7ff0ad7fa700 1 --2- 172.21.15.110:0/199940307 >> [v2:172.21.15.177:6802/22424,v1:172.21.15.177:6804/22424] conn(0x7ff0a006ebd0 0x7ff0a0070fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:53.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.839+0000 7ff0ad7fa700 1 -- 172.21.15.110:0/199940307 --> [v2:172.21.15.177:6802/22424,v1:172.21.15.177:6804/22424] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff0a0071690 con 0x7ff0a006ebd0 2024-09-15T03:40:53.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.839+0000 7ff0ad7fa700 1 -- 172.21.15.110:0/199940307 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0b40889a0 con 0x7ff0bc0b0af0 2024-09-15T03:40:53.844 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.839+0000 7ff0c0fbb700 1 --2- 172.21.15.110:0/199940307 >> [v2:172.21.15.177:6802/22424,v1:172.21.15.177:6804/22424] conn(0x7ff0a006ebd0 0x7ff0a0070fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:53.844 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.839+0000 7ff0c17bc700 1 --2- 172.21.15.110:0/199940307 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7ff0a0067a80 0x7ff0a0069f30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff0b00078a0 tx=0x7ff0b0007830 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:53.844 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.839+0000 7ff0c0fbb700 1 --2- 172.21.15.110:0/199940307 >> [v2:172.21.15.177:6802/22424,v1:172.21.15.177:6804/22424] conn(0x7ff0a006ebd0 0x7ff0a0070fe0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:53.845 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.843+0000 7ff0ad7fa700 1 -- 172.21.15.110:0/199940307 <== osd.5 v2:172.21.15.177:6802/22424 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7ff0a0071690 con 0x7ff0a006ebd0 2024-09-15T03:40:53.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.859+0000 7ff0c27be700 1 -- 172.21.15.110:0/199940307 --> [v2:172.21.15.177:6802/22424,v1:172.21.15.177:6804/22424] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff0bc14dae0 con 0x7ff0a006ebd0 2024-09-15T03:40:53.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.859+0000 7ff0ad7fa700 1 -- 172.21.15.110:0/199940307 <== osd.5 v2:172.21.15.177:6802/22424 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7ff0bc14dae0 con 0x7ff0a006ebd0 2024-09-15T03:40:53.863 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-15T03:40:53.863 INFO:teuthology.orchestra.run.smithi110.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T03:40:53.863 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-15T03:40:53.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.859+0000 7ff0c27be700 1 -- 172.21.15.110:0/199940307 >> [v2:172.21.15.177:6802/22424,v1:172.21.15.177:6804/22424] conn(0x7ff0a006ebd0 msgr2=0x7ff0a0070fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:53.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.859+0000 7ff0c27be700 1 --2- 172.21.15.110:0/199940307 >> [v2:172.21.15.177:6802/22424,v1:172.21.15.177:6804/22424] conn(0x7ff0a006ebd0 0x7ff0a0070fe0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:53.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.859+0000 7ff0c27be700 1 -- 172.21.15.110:0/199940307 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7ff0a0067a80 msgr2=0x7ff0a0069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:53.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.859+0000 7ff0c27be700 1 --2- 172.21.15.110:0/199940307 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7ff0a0067a80 0x7ff0a0069f30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff0b00078a0 tx=0x7ff0b0007830 comp rx=0 tx=0).stop 2024-09-15T03:40:53.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.859+0000 7ff0c27be700 1 -- 172.21.15.110:0/199940307 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff0bc0b0af0 msgr2=0x7ff0bc0abc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:53.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.859+0000 7ff0c27be700 1 --2- 172.21.15.110:0/199940307 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff0bc0b0af0 0x7ff0bc0abc20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4016000 tx=0x7ff0b4019000 comp rx=0 tx=0).stop 2024-09-15T03:40:53.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.859+0000 7ff0c1fbd700 1 -- 172.21.15.110:0/199940307 reap_dead start 2024-09-15T03:40:53.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.859+0000 7ff0c27be700 1 -- 172.21.15.110:0/199940307 shutdown_connections 2024-09-15T03:40:53.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.859+0000 7ff0c27be700 1 -- 172.21.15.110:0/199940307 >> 172.21.15.110:0/199940307 conn(0x7ff0bc069120 msgr2=0x7ff0bc13f080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:40:53.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.859+0000 7ff0c27be700 1 -- 172.21.15.110:0/199940307 shutdown_connections 2024-09-15T03:40:53.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.859+0000 7ff0c27be700 1 -- 172.21.15.110:0/199940307 wait complete. 2024-09-15T03:40:53.875 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-15T03:40:53.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.975+0000 7f709b9d6700 1 Processor -- start 2024-09-15T03:40:53.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.975+0000 7f709b9d6700 1 -- start start 2024-09-15T03:40:53.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.975+0000 7f709b9d6700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f709413a490 0x7f709413af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:53.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.975+0000 7f709b9d6700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f709406d780 0x7f709406dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:53.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.975+0000 7f709b9d6700 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f709413b7b0 con 0x7f7094063e40 2024-09-15T03:40:53.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.975+0000 7f709b9d6700 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f709413a8f0 con 0x7f709406d780 2024-09-15T03:40:53.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.975+0000 7f709b9d6700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f709413bc40 con 0x7f709413a490 2024-09-15T03:40:53.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.975+0000 7f7099772700 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f7094063e40 0x7f7094139d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:32924/0 (socket says 172.21.15.110:32924) 2024-09-15T03:40:53.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.975+0000 7f7099772700 1 -- 172.21.15.110:0/3644701214 learned_addr learned my addr 172.21.15.110:0/3644701214 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-15T03:40:53.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.975+0000 7f7099f73700 1 --2- 172.21.15.110:0/3644701214 >> v2:172.21.15.110:3300/0 conn(0x7f709406d780 0x7f709406dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:53.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.975+0000 7f7098f71700 1 --2- 172.21.15.110:0/3644701214 >> v2:172.21.15.177:3300/0 conn(0x7f709413a490 0x7f709413af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:53.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.975+0000 7f7099f73700 1 -- 172.21.15.110:0/3644701214 >> v2:172.21.15.177:3300/0 conn(0x7f709413a490 msgr2=0x7f709413af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:40:53.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.975+0000 7f7099f73700 1 --2- 172.21.15.110:0/3644701214 >> v2:172.21.15.177:3300/0 conn(0x7f709413a490 0x7f709413af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:53.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.975+0000 7f7099f73700 1 -- 172.21.15.110:0/3644701214 >> v1:172.21.15.110:6789/0 conn(0x7f7094063e40 legacy=0x7f7094139d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:53.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.975+0000 7f7099f73700 1 -- 172.21.15.110:0/3644701214 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f709413b4c0 con 0x7f709406d780 2024-09-15T03:40:53.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.975+0000 7f7099f73700 1 --2- 172.21.15.110:0/3644701214 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f709406d780 0x7f709406dbe0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f70940ae640 tx=0x7f708c00d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f053ccf65a38d04 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:53.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.975+0000 7f708bfff700 1 -- 172.21.15.110:0/3644701214 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708c00b2e0 con 0x7f709406d780 2024-09-15T03:40:53.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.979+0000 7f708bfff700 1 -- 172.21.15.110:0/3644701214 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f708c00c9d0 con 0x7f709406d780 2024-09-15T03:40:53.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.979+0000 7f708bfff700 1 -- 172.21.15.110:0/3644701214 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708c00cd90 con 0x7f709406d780 2024-09-15T03:40:53.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.979+0000 7f709b9d6700 1 -- 172.21.15.110:0/3644701214 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f709406d780 msgr2=0x7f709406dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:53.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.979+0000 7f709b9d6700 1 --2- 172.21.15.110:0/3644701214 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f709406d780 0x7f709406dbe0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f70940ae640 tx=0x7f708c00d040 comp rx=0 tx=0).stop 2024-09-15T03:40:53.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.979+0000 7f709b9d6700 1 -- 172.21.15.110:0/3644701214 shutdown_connections 2024-09-15T03:40:53.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.979+0000 7f709b9d6700 1 --2- 172.21.15.110:0/3644701214 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f709406d780 0x7f709406dbe0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:53.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.979+0000 7f709b9d6700 1 --2- 172.21.15.110:0/3644701214 >> v2:172.21.15.177:3300/0 conn(0x7f709413a490 0x7f709413af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:53.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.979+0000 7f709b9d6700 1 -- 172.21.15.110:0/3644701214 >> 172.21.15.110:0/3644701214 conn(0x7f70940b17e0 msgr2=0x7f70940ad840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:40:53.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.979+0000 7f709b9d6700 1 -- 172.21.15.110:0/3644701214 shutdown_connections 2024-09-15T03:40:53.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.979+0000 7f709b9d6700 1 -- 172.21.15.110:0/3644701214 wait complete. 2024-09-15T03:40:53.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.983+0000 7f709b9d6700 1 Processor -- start 2024-09-15T03:40:53.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.983+0000 7f709b9d6700 1 -- start start 2024-09-15T03:40:53.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.983+0000 7f709b9d6700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7094063e40 0x7f70940acac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:53.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.983+0000 7f709b9d6700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f709406d780 0x7f70940a84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:53.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.983+0000 7f709b9d6700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f709413a490 0x7f70940a89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:53.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.983+0000 7f709b9d6700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f709413e050 con 0x7f7094063e40 2024-09-15T03:40:53.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.983+0000 7f709b9d6700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f709413e4e0 con 0x7f709413a490 2024-09-15T03:40:53.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.983+0000 7f709b9d6700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f709413dd70 con 0x7f709406d780 2024-09-15T03:40:53.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.983+0000 7f7099772700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7094063e40 0x7f70940acac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:53.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.983+0000 7f7098f71700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f709406d780 0x7f70940a84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:53.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.983+0000 7f7099772700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7094063e40 0x7f70940acac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51278/0 (socket says 172.21.15.110:51278) 2024-09-15T03:40:53.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.983+0000 7f7099772700 1 -- 172.21.15.110:0/89839726 learned_addr learned my addr 172.21.15.110:0/89839726 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:40:53.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.983+0000 7f7099f73700 1 --2- 172.21.15.110:0/89839726 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f709413a490 0x7f70940a89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:53.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.983+0000 7f7098f71700 1 -- 172.21.15.110:0/89839726 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f709413a490 msgr2=0x7f70940a89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:53.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.983+0000 7f7098f71700 1 --2- 172.21.15.110:0/89839726 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f709413a490 0x7f70940a89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:53.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.983+0000 7f7098f71700 1 -- 172.21.15.110:0/89839726 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7094063e40 msgr2=0x7f70940acac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:53.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.983+0000 7f7098f71700 1 --2- 172.21.15.110:0/89839726 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7094063e40 0x7f70940acac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:53.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.983+0000 7f7098f71700 1 -- 172.21.15.110:0/89839726 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f708c00ba40 con 0x7f709406d780 2024-09-15T03:40:53.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.983+0000 7f7099772700 1 --2- 172.21.15.110:0/89839726 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7094063e40 0x7f70940acac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T03:40:53.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.983+0000 7f7098f71700 1 --2- 172.21.15.110:0/89839726 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f709406d780 0x7f70940a84a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7084012ca0 tx=0x7f708400f040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:53.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.983+0000 7f70897fa700 1 -- 172.21.15.110:0/89839726 <== mon.2 v2:172.21.15.110:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708400d2e0 con 0x7f709406d780 2024-09-15T03:40:53.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.983+0000 7f70897fa700 1 -- 172.21.15.110:0/89839726 <== mon.2 v2:172.21.15.110:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7084013040 con 0x7f709406d780 2024-09-15T03:40:53.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.983+0000 7f70897fa700 1 -- 172.21.15.110:0/89839726 <== mon.2 v2:172.21.15.110:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7084010900 con 0x7f709406d780 2024-09-15T03:40:53.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.983+0000 7f709b9d6700 1 -- 172.21.15.110:0/89839726 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70941370a0 con 0x7f709406d780 2024-09-15T03:40:53.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.983+0000 7f709b9d6700 1 -- 172.21.15.110:0/89839726 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70940a7e40 con 0x7f709406d780 2024-09-15T03:40:53.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.987+0000 7f709b9d6700 1 -- 172.21.15.110:0/89839726 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70940a8000 con 0x7f709406d780 2024-09-15T03:40:53.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.987+0000 7f70897fa700 1 -- 172.21.15.110:0/89839726 <== mon.2 v2:172.21.15.110:3301/0 4 ==== mgrmap(e 5) v1 ==== 85847+0+0 (secure 0 0 0) 0x7f7084010cb0 con 0x7f709406d780 2024-09-15T03:40:53.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.987+0000 7f70897fa700 1 --2- 172.21.15.110:0/89839726 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f7078067a80 0x7f7078069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:53.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.987+0000 7f7099772700 1 --2- 172.21.15.110:0/89839726 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f7078067a80 0x7f7078069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:53.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.987+0000 7f70897fa700 1 -- 172.21.15.110:0/89839726 <== mon.2 v2:172.21.15.110:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5641+0+0 (secure 0 0 0) 0x7f7084089700 con 0x7f709406d780 2024-09-15T03:40:53.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.987+0000 7f7099772700 1 --2- 172.21.15.110:0/89839726 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f7078067a80 0x7f7078069f30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7090000c00 tx=0x7f7090009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:53.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:53.995+0000 7f70897fa700 1 -- 172.21.15.110:0/89839726 <== mon.2 v2:172.21.15.110:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f7084056840 con 0x7f709406d780 2024-09-15T03:40:54.158 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.155+0000 7f709b9d6700 1 -- 172.21.15.110:0/89839726 --> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f70940721b0 con 0x7f7078067a80 2024-09-15T03:40:54.158 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.155+0000 7f70897fa700 1 -- 172.21.15.110:0/89839726 <== mgr.4104 v2:172.21.15.110:6832/22190 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f70940721b0 con 0x7f7078067a80 2024-09-15T03:40:54.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.159+0000 7f709b9d6700 1 -- 172.21.15.110:0/89839726 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f7078067a80 msgr2=0x7f7078069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:54.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.159+0000 7f709b9d6700 1 --2- 172.21.15.110:0/89839726 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f7078067a80 0x7f7078069f30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7090000c00 tx=0x7f7090009040 comp rx=0 tx=0).stop 2024-09-15T03:40:54.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.159+0000 7f709b9d6700 1 -- 172.21.15.110:0/89839726 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f709406d780 msgr2=0x7f70940a84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:54.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.159+0000 7f709b9d6700 1 --2- 172.21.15.110:0/89839726 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f709406d780 0x7f70940a84a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7084012ca0 tx=0x7f708400f040 comp rx=0 tx=0).stop 2024-09-15T03:40:54.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.159+0000 7f709b9d6700 1 -- 172.21.15.110:0/89839726 shutdown_connections 2024-09-15T03:40:54.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.159+0000 7f709b9d6700 1 --2- 172.21.15.110:0/89839726 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f7078067a80 0x7f7078069f30 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:54.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.159+0000 7f709b9d6700 1 --2- 172.21.15.110:0/89839726 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f709413a490 0x7f70940a89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:54.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.159+0000 7f709b9d6700 1 --2- 172.21.15.110:0/89839726 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f709406d780 0x7f70940a84a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:54.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.159+0000 7f709b9d6700 1 --2- 172.21.15.110:0/89839726 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7094063e40 0x7f70940acac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:54.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.159+0000 7f709b9d6700 1 -- 172.21.15.110:0/89839726 >> 172.21.15.110:0/89839726 conn(0x7f70940b17e0 msgr2=0x7f70940afd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:40:54.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.159+0000 7f709b9d6700 1 -- 172.21.15.110:0/89839726 shutdown_connections 2024-09-15T03:40:54.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.159+0000 7f709b9d6700 1 -- 172.21.15.110:0/89839726 wait complete. 2024-09-15T03:40:54.164 INFO:teuthology.orchestra.run.smithi110.stderr:instructed osd(s) 5 to deep-scrub 2024-09-15T03:40:54.176 INFO:tasks.ceph:Scrubbing osd.6 2024-09-15T03:40:54.177 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-15T03:40:54.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.279+0000 7f8851313700 1 Processor -- start 2024-09-15T03:40:54.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.279+0000 7f8851313700 1 -- start start 2024-09-15T03:40:54.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.283+0000 7f8851313700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f884c0b1e40 0x7f884c13df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:54.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.283+0000 7f8851313700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f884c13a330 0x7f884c1452f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:54.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.283+0000 7f8851313700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f884c06f100 con 0x7f884c13e490 2024-09-15T03:40:54.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.283+0000 7f8851313700 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f884c13a790 con 0x7f884c0b1e40 2024-09-15T03:40:54.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.283+0000 7f8851313700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f884c0b2210 con 0x7f884c13a330 2024-09-15T03:40:54.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.283+0000 7f884bfff700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f884c0b1e40 0x7f884c13df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:54.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.283+0000 7f8850b12700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f884c13a330 0x7f884c1452f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:54.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.283+0000 7f884bfff700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f884c0b1e40 0x7f884c13df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51282/0 (socket says 172.21.15.110:51282) 2024-09-15T03:40:54.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.283+0000 7f884bfff700 1 -- 172.21.15.110:0/3661219921 learned_addr learned my addr 172.21.15.110:0/3661219921 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:40:54.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.283+0000 7f884bfff700 1 -- 172.21.15.110:0/3661219921 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f884c13a330 msgr2=0x7f884c1452f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:40:54.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.283+0000 7f884bfff700 1 --2- 172.21.15.110:0/3661219921 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f884c13a330 0x7f884c1452f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:54.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.283+0000 7f884bfff700 1 -- 172.21.15.110:0/3661219921 >> v1:172.21.15.177:6789/0 conn(0x7f884c13e490 legacy=0x7f884c139c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:54.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.283+0000 7f884bfff700 1 -- 172.21.15.110:0/3661219921 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f884c0a5500 con 0x7f884c0b1e40 2024-09-15T03:40:54.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.283+0000 7f884bfff700 1 --2- 172.21.15.110:0/3661219921 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f884c0b1e40 0x7f884c13df50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f883c0099b0 tx=0x7f883c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26651a96d5d55daf server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:54.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.287+0000 7f884affd700 1 -- 172.21.15.110:0/3661219921 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883c009ea0 con 0x7f884c0b1e40 2024-09-15T03:40:54.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.287+0000 7f884affd700 1 -- 172.21.15.110:0/3661219921 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f883c00b860 con 0x7f884c0b1e40 2024-09-15T03:40:54.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.287+0000 7f884affd700 1 -- 172.21.15.110:0/3661219921 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883c00bc00 con 0x7f884c0b1e40 2024-09-15T03:40:54.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.287+0000 7f8851313700 1 -- 172.21.15.110:0/3661219921 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f884c0b1e40 msgr2=0x7f884c13df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:54.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.287+0000 7f8851313700 1 --2- 172.21.15.110:0/3661219921 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f884c0b1e40 0x7f884c13df50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f883c0099b0 tx=0x7f883c009250 comp rx=0 tx=0).stop 2024-09-15T03:40:54.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.287+0000 7f8851313700 1 -- 172.21.15.110:0/3661219921 shutdown_connections 2024-09-15T03:40:54.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.287+0000 7f8851313700 1 --2- 172.21.15.110:0/3661219921 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f884c13a330 0x7f884c1452f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:54.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.287+0000 7f8851313700 1 --2- 172.21.15.110:0/3661219921 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f884c0b1e40 0x7f884c13df50 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:54.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.287+0000 7f8851313700 1 -- 172.21.15.110:0/3661219921 >> 172.21.15.110:0/3661219921 conn(0x7f884c0680f0 msgr2=0x7f884c0b1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:40:54.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.287+0000 7f8851313700 1 -- 172.21.15.110:0/3661219921 shutdown_connections 2024-09-15T03:40:54.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.287+0000 7f8851313700 1 -- 172.21.15.110:0/3661219921 wait complete. 2024-09-15T03:40:54.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.287+0000 7f8851313700 1 Processor -- start 2024-09-15T03:40:54.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.291+0000 7f8851313700 1 -- start start 2024-09-15T03:40:54.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.291+0000 7f8851313700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f884c0b1e40 0x7f884c13d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:54.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.291+0000 7f8851313700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f884c13a330 0x7f884c0a7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:54.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.291+0000 7f8851313700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f884c13e490 0x7f884c0ac5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:54.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.291+0000 7f8851313700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f884c13e080 con 0x7f884c13a330 2024-09-15T03:40:54.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.291+0000 7f8851313700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f884c135450 con 0x7f884c0b1e40 2024-09-15T03:40:54.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.291+0000 7f8851313700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f884c0a7e80 con 0x7f884c13e490 2024-09-15T03:40:54.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.291+0000 7f884b7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f884c13a330 0x7f884c0a7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:54.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.291+0000 7f8850b12700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f884c13e490 0x7f884c0ac5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:54.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.291+0000 7f8850b12700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f884c13e490 0x7f884c0ac5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3301/0 says I am v2:172.21.15.110:43274/0 (socket says 172.21.15.110:43274) 2024-09-15T03:40:54.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.291+0000 7f884b7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f884c13a330 0x7f884c0a7590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51284/0 (socket says 172.21.15.110:51284) 2024-09-15T03:40:54.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.291+0000 7f8850b12700 1 -- 172.21.15.110:0/1074702119 learned_addr learned my addr 172.21.15.110:0/1074702119 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:40:54.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.291+0000 7f884bfff700 1 --2- 172.21.15.110:0/1074702119 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f884c0b1e40 0x7f884c13d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:54.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.291+0000 7f884b7fe700 1 -- 172.21.15.110:0/1074702119 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f884c13e490 msgr2=0x7f884c0ac5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:54.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.291+0000 7f884b7fe700 1 --2- 172.21.15.110:0/1074702119 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f884c13e490 0x7f884c0ac5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:54.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.291+0000 7f884b7fe700 1 -- 172.21.15.110:0/1074702119 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f884c0b1e40 msgr2=0x7f884c13d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:54.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.291+0000 7f884b7fe700 1 --2- 172.21.15.110:0/1074702119 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f884c0b1e40 0x7f884c13d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:54.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.291+0000 7f884b7fe700 1 -- 172.21.15.110:0/1074702119 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f883c0094f0 con 0x7f884c13a330 2024-09-15T03:40:54.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.291+0000 7f8850b12700 1 --2- 172.21.15.110:0/1074702119 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f884c13e490 0x7f884c0ac5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T03:40:54.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.291+0000 7f884b7fe700 1 --2- 172.21.15.110:0/1074702119 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f884c13a330 0x7f884c0a7590 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f883400c490 tx=0x7f8834007ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:54.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.291+0000 7f8843fff700 1 -- 172.21.15.110:0/1074702119 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883400ce40 con 0x7f884c13a330 2024-09-15T03:40:54.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.291+0000 7f8843fff700 1 -- 172.21.15.110:0/1074702119 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8834011590 con 0x7f884c13a330 2024-09-15T03:40:54.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.291+0000 7f8851313700 1 -- 172.21.15.110:0/1074702119 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f884c0a8a00 con 0x7f884c13a330 2024-09-15T03:40:54.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.291+0000 7f8843fff700 1 -- 172.21.15.110:0/1074702119 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8834011950 con 0x7f884c13a330 2024-09-15T03:40:54.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.291+0000 7f8851313700 1 -- 172.21.15.110:0/1074702119 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f884c0acb10 con 0x7f884c13a330 2024-09-15T03:40:54.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.291+0000 7f8851313700 1 -- 172.21.15.110:0/1074702119 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f884c14b6e0 con 0x7f884c13a330 2024-09-15T03:40:54.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.295+0000 7f8843fff700 1 -- 172.21.15.110:0/1074702119 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) v1 ==== 85847+0+0 (secure 0 0 0) 0x7f8834011d00 con 0x7f884c13a330 2024-09-15T03:40:54.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.295+0000 7f8843fff700 1 --2- 172.21.15.110:0/1074702119 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f882c067ab0 0x7f882c069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:54.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.295+0000 7f8843fff700 1 -- 172.21.15.110:0/1074702119 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5641+0+0 (secure 0 0 0) 0x7f8834084940 con 0x7f884c13a330 2024-09-15T03:40:54.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.295+0000 7f884bfff700 1 --2- 172.21.15.110:0/1074702119 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f882c067ab0 0x7f882c069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:54.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.295+0000 7f8843fff700 1 --2- 172.21.15.110:0/1074702119 >> [v2:172.21.15.177:6816/22428,v1:172.21.15.177:6817/22428] conn(0x7f882c06ec00 0x7f882c071010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:54.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.295+0000 7f8843fff700 1 -- 172.21.15.110:0/1074702119 --> [v2:172.21.15.177:6816/22428,v1:172.21.15.177:6817/22428] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f882c0716c0 con 0x7f882c06ec00 2024-09-15T03:40:54.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.295+0000 7f8843fff700 1 -- 172.21.15.110:0/1074702119 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8834089220 con 0x7f884c13a330 2024-09-15T03:40:54.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.295+0000 7f8850b12700 1 --2- 172.21.15.110:0/1074702119 >> [v2:172.21.15.177:6816/22428,v1:172.21.15.177:6817/22428] conn(0x7f882c06ec00 0x7f882c071010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:54.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.295+0000 7f884bfff700 1 --2- 172.21.15.110:0/1074702119 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f882c067ab0 0x7f882c069f60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f883c0099b0 tx=0x7f883c019000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:54.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.295+0000 7f8850b12700 1 --2- 172.21.15.110:0/1074702119 >> [v2:172.21.15.177:6816/22428,v1:172.21.15.177:6817/22428] conn(0x7f882c06ec00 0x7f882c071010 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:54.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.299+0000 7f8843fff700 1 -- 172.21.15.110:0/1074702119 <== osd.6 v2:172.21.15.177:6816/22428 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f882c0716c0 con 0x7f882c06ec00 2024-09-15T03:40:54.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.315+0000 7f8851313700 1 -- 172.21.15.110:0/1074702119 --> [v2:172.21.15.177:6816/22428,v1:172.21.15.177:6817/22428] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f884c0a7c40 con 0x7f882c06ec00 2024-09-15T03:40:54.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.315+0000 7f8843fff700 1 -- 172.21.15.110:0/1074702119 <== osd.6 v2:172.21.15.177:6816/22428 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f884c0a7c40 con 0x7f882c06ec00 2024-09-15T03:40:54.318 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-15T03:40:54.318 INFO:teuthology.orchestra.run.smithi110.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T03:40:54.318 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-15T03:40:54.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.315+0000 7f8851313700 1 -- 172.21.15.110:0/1074702119 >> [v2:172.21.15.177:6816/22428,v1:172.21.15.177:6817/22428] conn(0x7f882c06ec00 msgr2=0x7f882c071010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:54.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.315+0000 7f8851313700 1 --2- 172.21.15.110:0/1074702119 >> [v2:172.21.15.177:6816/22428,v1:172.21.15.177:6817/22428] conn(0x7f882c06ec00 0x7f882c071010 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:54.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.315+0000 7f8851313700 1 -- 172.21.15.110:0/1074702119 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f882c067ab0 msgr2=0x7f882c069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:54.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.315+0000 7f8851313700 1 --2- 172.21.15.110:0/1074702119 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f882c067ab0 0x7f882c069f60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f883c0099b0 tx=0x7f883c019000 comp rx=0 tx=0).stop 2024-09-15T03:40:54.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.315+0000 7f8851313700 1 -- 172.21.15.110:0/1074702119 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f884c13a330 msgr2=0x7f884c0a7590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:54.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.315+0000 7f8851313700 1 --2- 172.21.15.110:0/1074702119 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f884c13a330 0x7f884c0a7590 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f883400c490 tx=0x7f8834007ec0 comp rx=0 tx=0).stop 2024-09-15T03:40:54.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.315+0000 7f8850b12700 1 -- 172.21.15.110:0/1074702119 reap_dead start 2024-09-15T03:40:54.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.315+0000 7f8851313700 1 -- 172.21.15.110:0/1074702119 shutdown_connections 2024-09-15T03:40:54.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.315+0000 7f8851313700 1 -- 172.21.15.110:0/1074702119 >> 172.21.15.110:0/1074702119 conn(0x7f884c0680f0 msgr2=0x7f884c136f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:40:54.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.315+0000 7f8851313700 1 -- 172.21.15.110:0/1074702119 shutdown_connections 2024-09-15T03:40:54.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.315+0000 7f8851313700 1 -- 172.21.15.110:0/1074702119 wait complete. 2024-09-15T03:40:54.330 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-15T03:40:54.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.447+0000 7ff10988c700 1 Processor -- start 2024-09-15T03:40:54.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.447+0000 7ff10988c700 1 -- start start 2024-09-15T03:40:54.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.447+0000 7ff10988c700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ff1040b2a70 0x7ff10413ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:54.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.447+0000 7ff10988c700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7ff1040b2e40 0x7ff1040ae590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:54.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.447+0000 7ff10988c700 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff10413f1e0 con 0x7ff10413f670 2024-09-15T03:40:54.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.447+0000 7ff10988c700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff10413bf60 con 0x7ff1040b2a70 2024-09-15T03:40:54.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.447+0000 7ff10988c700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1040a5b40 con 0x7ff1040b2e40 2024-09-15T03:40:54.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.447+0000 7ff10908b700 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7ff10413f670 0x7ff1040a4ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:32934/0 (socket says 172.21.15.110:32934) 2024-09-15T03:40:54.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.447+0000 7ff10908b700 1 -- 172.21.15.110:0/3487865969 learned_addr learned my addr 172.21.15.110:0/3487865969 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-15T03:40:54.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.447+0000 7ff103fff700 1 --2- 172.21.15.110:0/3487865969 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7ff1040b2e40 0x7ff1040ae590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:54.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.447+0000 7ff10888a700 1 --2- 172.21.15.110:0/3487865969 >> v2:172.21.15.177:3300/0 conn(0x7ff1040b2a70 0x7ff10413ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:54.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.447+0000 7ff103fff700 1 -- 172.21.15.110:0/3487865969 >> v2:172.21.15.177:3300/0 conn(0x7ff1040b2a70 msgr2=0x7ff10413ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:40:54.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.447+0000 7ff103fff700 1 --2- 172.21.15.110:0/3487865969 >> v2:172.21.15.177:3300/0 conn(0x7ff1040b2a70 0x7ff10413ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:54.453 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.447+0000 7ff1037fe700 1 -- 172.21.15.110:0/3487865969 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2047307253 0 0) 0x7ff10413f1e0 con 0x7ff10413f670 2024-09-15T03:40:54.453 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.447+0000 7ff103fff700 1 -- 172.21.15.110:0/3487865969 >> v1:172.21.15.110:6789/0 conn(0x7ff10413f670 legacy=0x7ff1040a4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:54.453 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.447+0000 7ff103fff700 1 -- 172.21.15.110:0/3487865969 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1040a6b00 con 0x7ff1040b2e40 2024-09-15T03:40:54.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.447+0000 7ff103fff700 1 --2- 172.21.15.110:0/3487865969 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7ff1040b2e40 0x7ff1040ae590 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff0f40098e0 tx=0x7ff0f40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=668dca133365d3ee server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:54.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.447+0000 7ff1037fe700 1 -- 172.21.15.110:0/3487865969 <== mon.2 v2:172.21.15.110:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f400b640 con 0x7ff1040b2e40 2024-09-15T03:40:54.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.447+0000 7ff1037fe700 1 -- 172.21.15.110:0/3487865969 <== mon.2 v2:172.21.15.110:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0f400b7a0 con 0x7ff1040b2e40 2024-09-15T03:40:54.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.447+0000 7ff1037fe700 1 -- 172.21.15.110:0/3487865969 <== mon.2 v2:172.21.15.110:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f400bb40 con 0x7ff1040b2e40 2024-09-15T03:40:54.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.451+0000 7ff10988c700 1 -- 172.21.15.110:0/3487865969 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7ff1040b2e40 msgr2=0x7ff1040ae590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:54.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.451+0000 7ff10988c700 1 --2- 172.21.15.110:0/3487865969 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7ff1040b2e40 0x7ff1040ae590 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff0f40098e0 tx=0x7ff0f40091a0 comp rx=0 tx=0).stop 2024-09-15T03:40:54.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.451+0000 7ff10988c700 1 -- 172.21.15.110:0/3487865969 shutdown_connections 2024-09-15T03:40:54.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.451+0000 7ff10988c700 1 --2- 172.21.15.110:0/3487865969 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7ff1040b2e40 0x7ff1040ae590 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:54.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.451+0000 7ff10988c700 1 --2- 172.21.15.110:0/3487865969 >> v2:172.21.15.177:3300/0 conn(0x7ff1040b2a70 0x7ff10413ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:54.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.451+0000 7ff10988c700 1 -- 172.21.15.110:0/3487865969 >> 172.21.15.110:0/3487865969 conn(0x7ff1040680f0 msgr2=0x7ff1040b2360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:40:54.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.451+0000 7ff10988c700 1 -- 172.21.15.110:0/3487865969 shutdown_connections 2024-09-15T03:40:54.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.451+0000 7ff10988c700 1 -- 172.21.15.110:0/3487865969 wait complete. 2024-09-15T03:40:54.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.451+0000 7ff10988c700 1 Processor -- start 2024-09-15T03:40:54.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.451+0000 7ff10988c700 1 -- start start 2024-09-15T03:40:54.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.451+0000 7ff10988c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1040b2a70 0x7ff10413df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:54.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.451+0000 7ff10988c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff1040b2e40 0x7ff1040a7750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:54.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.451+0000 7ff10988c700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7ff10413f670 0x7ff104137c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:54.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.451+0000 7ff10988c700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff104138160 con 0x7ff1040b2e40 2024-09-15T03:40:54.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.451+0000 7ff10988c700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff104138380 con 0x7ff1040b2a70 2024-09-15T03:40:54.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.451+0000 7ff10988c700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff10414e810 con 0x7ff10413f670 2024-09-15T03:40:54.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.451+0000 7ff103fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff1040b2e40 0x7ff1040a7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:54.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.451+0000 7ff10908b700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7ff10413f670 0x7ff104137c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:54.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.451+0000 7ff103fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff1040b2e40 0x7ff1040a7750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51296/0 (socket says 172.21.15.110:51296) 2024-09-15T03:40:54.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.451+0000 7ff10888a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1040b2a70 0x7ff10413df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:54.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.451+0000 7ff103fff700 1 -- 172.21.15.110:0/3797021955 learned_addr learned my addr 172.21.15.110:0/3797021955 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:40:54.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.451+0000 7ff10908b700 1 -- 172.21.15.110:0/3797021955 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1040b2a70 msgr2=0x7ff10413df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:54.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.451+0000 7ff10908b700 1 --2- 172.21.15.110:0/3797021955 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1040b2a70 0x7ff10413df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:54.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.451+0000 7ff10908b700 1 -- 172.21.15.110:0/3797021955 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff1040b2e40 msgr2=0x7ff1040a7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:54.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.451+0000 7ff10908b700 1 --2- 172.21.15.110:0/3797021955 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff1040b2e40 0x7ff1040a7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:54.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.451+0000 7ff10908b700 1 -- 172.21.15.110:0/3797021955 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0f4009420 con 0x7ff10413f670 2024-09-15T03:40:54.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.451+0000 7ff10908b700 1 --2- 172.21.15.110:0/3797021955 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7ff10413f670 0x7ff104137c20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc00cbc0 tx=0x7ff0fc0018f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:54.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.451+0000 7ff100ff9700 1 -- 172.21.15.110:0/3797021955 <== mon.2 v2:172.21.15.110:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc00a740 con 0x7ff10413f670 2024-09-15T03:40:54.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.455+0000 7ff100ff9700 1 -- 172.21.15.110:0/3797021955 <== mon.2 v2:172.21.15.110:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0fc00a8a0 con 0x7ff10413f670 2024-09-15T03:40:54.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.455+0000 7ff10988c700 1 -- 172.21.15.110:0/3797021955 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff10414ea90 con 0x7ff10413f670 2024-09-15T03:40:54.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.455+0000 7ff100ff9700 1 -- 172.21.15.110:0/3797021955 <== mon.2 v2:172.21.15.110:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc00ac60 con 0x7ff10413f670 2024-09-15T03:40:54.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.455+0000 7ff10988c700 1 -- 172.21.15.110:0/3797021955 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff10414efb0 con 0x7ff10413f670 2024-09-15T03:40:54.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.455+0000 7ff10988c700 1 -- 172.21.15.110:0/3797021955 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1040721b0 con 0x7ff10413f670 2024-09-15T03:40:54.459 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.455+0000 7ff100ff9700 1 -- 172.21.15.110:0/3797021955 <== mon.2 v2:172.21.15.110:3301/0 4 ==== mgrmap(e 5) v1 ==== 85847+0+0 (secure 0 0 0) 0x7ff0fc007300 con 0x7ff10413f670 2024-09-15T03:40:54.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.455+0000 7ff100ff9700 1 --2- 172.21.15.110:0/3797021955 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7ff0f0067ab0 0x7ff0f0069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:54.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.455+0000 7ff100ff9700 1 -- 172.21.15.110:0/3797021955 <== mon.2 v2:172.21.15.110:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5641+0+0 (secure 0 0 0) 0x7ff0fc0858c0 con 0x7ff10413f670 2024-09-15T03:40:54.460 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.455+0000 7ff10888a700 1 --2- 172.21.15.110:0/3797021955 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7ff0f0067ab0 0x7ff0f0069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:54.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.455+0000 7ff10888a700 1 --2- 172.21.15.110:0/3797021955 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7ff0f0067ab0 0x7ff0f0069f60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8009db0 tx=0x7ff0f80091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:54.466 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.463+0000 7ff100ff9700 1 -- 172.21.15.110:0/3797021955 <== mon.2 v2:172.21.15.110:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7ff0fc052a00 con 0x7ff10413f670 2024-09-15T03:40:54.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.651+0000 7ff10988c700 1 -- 172.21.15.110:0/3797021955 --> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7ff1041384f0 con 0x7ff0f0067ab0 2024-09-15T03:40:54.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.651+0000 7ff100ff9700 1 -- 172.21.15.110:0/3797021955 <== mgr.4104 v2:172.21.15.110:6832/22190 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ff1041384f0 con 0x7ff0f0067ab0 2024-09-15T03:40:54.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.655+0000 7ff10988c700 1 -- 172.21.15.110:0/3797021955 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7ff0f0067ab0 msgr2=0x7ff0f0069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:54.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.655+0000 7ff10988c700 1 --2- 172.21.15.110:0/3797021955 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7ff0f0067ab0 0x7ff0f0069f60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8009db0 tx=0x7ff0f80091a0 comp rx=0 tx=0).stop 2024-09-15T03:40:54.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.655+0000 7ff10988c700 1 -- 172.21.15.110:0/3797021955 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7ff10413f670 msgr2=0x7ff104137c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:54.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.655+0000 7ff10988c700 1 --2- 172.21.15.110:0/3797021955 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7ff10413f670 0x7ff104137c20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc00cbc0 tx=0x7ff0fc0018f0 comp rx=0 tx=0).stop 2024-09-15T03:40:54.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.655+0000 7ff10988c700 1 -- 172.21.15.110:0/3797021955 shutdown_connections 2024-09-15T03:40:54.658 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.655+0000 7ff10988c700 1 --2- 172.21.15.110:0/3797021955 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7ff0f0067ab0 0x7ff0f0069f60 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:54.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.655+0000 7ff10988c700 1 --2- 172.21.15.110:0/3797021955 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7ff10413f670 0x7ff104137c20 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:54.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.655+0000 7ff10988c700 1 --2- 172.21.15.110:0/3797021955 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff1040b2e40 0x7ff1040a7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:54.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.655+0000 7ff10988c700 1 --2- 172.21.15.110:0/3797021955 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1040b2a70 0x7ff10413df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:54.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.655+0000 7ff10988c700 1 -- 172.21.15.110:0/3797021955 >> 172.21.15.110:0/3797021955 conn(0x7ff1040680f0 msgr2=0x7ff104138880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:40:54.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.655+0000 7ff10988c700 1 -- 172.21.15.110:0/3797021955 shutdown_connections 2024-09-15T03:40:54.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.655+0000 7ff10988c700 1 -- 172.21.15.110:0/3797021955 wait complete. 2024-09-15T03:40:54.659 INFO:teuthology.orchestra.run.smithi110.stderr:instructed osd(s) 6 to deep-scrub 2024-09-15T03:40:54.673 INFO:tasks.ceph:Scrubbing osd.7 2024-09-15T03:40:54.673 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-15T03:40:54.785 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.779+0000 7f5a1162d700 1 Processor -- start 2024-09-15T03:40:54.785 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.779+0000 7f5a1162d700 1 -- start start 2024-09-15T03:40:54.787 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.783+0000 7f5a1162d700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f5a0c0b18f0 0x7f5a0c06e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:54.787 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.783+0000 7f5a1162d700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f5a0c141b30 0x7f5a0c0b0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:54.787 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.783+0000 7f5a1162d700 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a0c1416a0 con 0x7f5a0c0b1cc0 2024-09-15T03:40:54.787 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.783+0000 7f5a1162d700 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a0c13dba0 con 0x7f5a0c141b30 2024-09-15T03:40:54.787 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.783+0000 7f5a1162d700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a0c141330 con 0x7f5a0c0b18f0 2024-09-15T03:40:54.787 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.783+0000 7f5a0b7fe700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f5a0c141b30 0x7f5a0c0b0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:54.787 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.783+0000 7f5a0a7fc700 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f5a0c0b1cc0 0x7f5a0c13de00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:32940/0 (socket says 172.21.15.110:32940) 2024-09-15T03:40:54.787 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.783+0000 7f5a0a7fc700 1 -- 172.21.15.110:0/3088106851 learned_addr learned my addr 172.21.15.110:0/3088106851 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-15T03:40:54.787 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.783+0000 7f5a0affd700 1 --2- 172.21.15.110:0/3088106851 >> v2:172.21.15.177:3300/0 conn(0x7f5a0c0b18f0 0x7f5a0c06e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:54.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.783+0000 7f5a0b7fe700 1 -- 172.21.15.110:0/3088106851 >> v2:172.21.15.177:3300/0 conn(0x7f5a0c0b18f0 msgr2=0x7f5a0c06e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:40:54.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.783+0000 7f5a0b7fe700 1 --2- 172.21.15.110:0/3088106851 >> v2:172.21.15.177:3300/0 conn(0x7f5a0c0b18f0 0x7f5a0c06e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:54.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.783+0000 7f5a0b7fe700 1 -- 172.21.15.110:0/3088106851 >> v1:172.21.15.110:6789/0 conn(0x7f5a0c0b1cc0 legacy=0x7f5a0c13de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:54.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.783+0000 7f5a09ffb700 1 -- 172.21.15.110:0/3088106851 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 881208810 0 0) 0x7f5a0c1416a0 con 0x7f5a0c0b1cc0 2024-09-15T03:40:54.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.783+0000 7f5a0b7fe700 1 -- 172.21.15.110:0/3088106851 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a0c06f0f0 con 0x7f5a0c141b30 2024-09-15T03:40:54.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.783+0000 7f5a0b7fe700 1 --2- 172.21.15.110:0/3088106851 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a0c141b30 0x7f5a0c0b0890 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c06b760 tx=0x7f5a04009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff2aea173835892b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:54.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.783+0000 7f5a09ffb700 1 -- 172.21.15.110:0/3088106851 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0400e070 con 0x7f5a0c141b30 2024-09-15T03:40:54.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.783+0000 7f5a09ffb700 1 -- 172.21.15.110:0/3088106851 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a0400bc30 con 0x7f5a0c141b30 2024-09-15T03:40:54.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.783+0000 7f5a09ffb700 1 -- 172.21.15.110:0/3088106851 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a04012430 con 0x7f5a0c141b30 2024-09-15T03:40:54.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.783+0000 7f5a1162d700 1 -- 172.21.15.110:0/3088106851 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a0c141b30 msgr2=0x7f5a0c0b0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:54.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.783+0000 7f5a1162d700 1 --2- 172.21.15.110:0/3088106851 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a0c141b30 0x7f5a0c0b0890 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c06b760 tx=0x7f5a04009780 comp rx=0 tx=0).stop 2024-09-15T03:40:54.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.783+0000 7f5a1162d700 1 -- 172.21.15.110:0/3088106851 shutdown_connections 2024-09-15T03:40:54.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.783+0000 7f5a1162d700 1 --2- 172.21.15.110:0/3088106851 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a0c141b30 0x7f5a0c0b0890 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:54.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.783+0000 7f5a1162d700 1 --2- 172.21.15.110:0/3088106851 >> v2:172.21.15.177:3300/0 conn(0x7f5a0c0b18f0 0x7f5a0c06e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:54.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.783+0000 7f5a1162d700 1 -- 172.21.15.110:0/3088106851 >> 172.21.15.110:0/3088106851 conn(0x7f5a0c067be0 msgr2=0x7f5a0c067fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:40:54.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.783+0000 7f5a1162d700 1 -- 172.21.15.110:0/3088106851 shutdown_connections 2024-09-15T03:40:54.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.783+0000 7f5a1162d700 1 -- 172.21.15.110:0/3088106851 wait complete. 2024-09-15T03:40:54.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.787+0000 7f5a1162d700 1 Processor -- start 2024-09-15T03:40:54.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.787+0000 7f5a1162d700 1 -- start start 2024-09-15T03:40:54.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.787+0000 7f5a1162d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5a0c0b18f0 0x7f5a0c0ae5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:54.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.787+0000 7f5a1162d700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f5a0c0b1cc0 0x7f5a0c0ad540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:54.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.787+0000 7f5a1162d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a0c141b30 0x7f5a0c14d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:54.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.787+0000 7f5a1162d700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0c14cf80 con 0x7f5a0c141b30 2024-09-15T03:40:54.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.787+0000 7f5a1162d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0c0ac970 con 0x7f5a0c0b18f0 2024-09-15T03:40:54.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.787+0000 7f5a1162d700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0c0aab30 con 0x7f5a0c0b1cc0 2024-09-15T03:40:54.793 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.787+0000 7f5a0b7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a0c141b30 0x7f5a0c14d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:54.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.787+0000 7f5a0a7fc700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f5a0c0b1cc0 0x7f5a0c0ad540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:54.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.787+0000 7f5a0b7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a0c141b30 0x7f5a0c14d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51310/0 (socket says 172.21.15.110:51310) 2024-09-15T03:40:54.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.787+0000 7f5a0b7fe700 1 -- 172.21.15.110:0/1216530700 learned_addr learned my addr 172.21.15.110:0/1216530700 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:40:54.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.787+0000 7f5a0affd700 1 --2- 172.21.15.110:0/1216530700 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5a0c0b18f0 0x7f5a0c0ae5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:54.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.787+0000 7f5a0b7fe700 1 -- 172.21.15.110:0/1216530700 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f5a0c0b1cc0 msgr2=0x7f5a0c0ad540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:54.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.787+0000 7f5a0b7fe700 1 --2- 172.21.15.110:0/1216530700 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f5a0c0b1cc0 0x7f5a0c0ad540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:54.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.787+0000 7f5a0b7fe700 1 -- 172.21.15.110:0/1216530700 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5a0c0b18f0 msgr2=0x7f5a0c0ae5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:54.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.787+0000 7f5a0b7fe700 1 --2- 172.21.15.110:0/1216530700 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5a0c0b18f0 0x7f5a0c0ae5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:54.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.787+0000 7f5a0b7fe700 1 -- 172.21.15.110:0/1216530700 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a04009a20 con 0x7f5a0c141b30 2024-09-15T03:40:54.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.787+0000 7f5a0b7fe700 1 --2- 172.21.15.110:0/1216530700 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a0c141b30 0x7f5a0c14d3d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c06da50 tx=0x7f5a04019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:54.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.787+0000 7f5a02ffd700 1 -- 172.21.15.110:0/1216530700 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a04027470 con 0x7f5a0c141b30 2024-09-15T03:40:54.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.787+0000 7f5a02ffd700 1 -- 172.21.15.110:0/1216530700 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a040275d0 con 0x7f5a0c141b30 2024-09-15T03:40:54.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.787+0000 7f5a1162d700 1 -- 172.21.15.110:0/1216530700 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a0c0aad50 con 0x7f5a0c141b30 2024-09-15T03:40:54.795 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.791+0000 7f5a02ffd700 1 -- 172.21.15.110:0/1216530700 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a04027990 con 0x7f5a0c141b30 2024-09-15T03:40:54.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.791+0000 7f5a1162d700 1 -- 172.21.15.110:0/1216530700 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a0c14d910 con 0x7f5a0c141b30 2024-09-15T03:40:54.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.791+0000 7f5a1162d700 1 -- 172.21.15.110:0/1216530700 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5a0c0adcf0 con 0x7f5a0c141b30 2024-09-15T03:40:54.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.791+0000 7f5a02ffd700 1 -- 172.21.15.110:0/1216530700 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) v1 ==== 85847+0+0 (secure 0 0 0) 0x7f5a04024070 con 0x7f5a0c141b30 2024-09-15T03:40:54.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.791+0000 7f5a02ffd700 1 --2- 172.21.15.110:0/1216530700 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f59ec067a80 0x7f59ec069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:54.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.791+0000 7f5a0affd700 1 --2- 172.21.15.110:0/1216530700 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f59ec067a80 0x7f59ec069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:54.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.791+0000 7f5a02ffd700 1 -- 172.21.15.110:0/1216530700 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5641+0+0 (secure 0 0 0) 0x7f5a04087ee0 con 0x7f5a0c141b30 2024-09-15T03:40:54.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.791+0000 7f5a02ffd700 1 --2- 172.21.15.110:0/1216530700 >> [v2:172.21.15.177:6800/22423,v1:172.21.15.177:6801/22423] conn(0x7f59ec06ebd0 0x7f59ec070fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:54.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.791+0000 7f5a02ffd700 1 -- 172.21.15.110:0/1216530700 --> [v2:172.21.15.177:6800/22423,v1:172.21.15.177:6801/22423] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f59ec071690 con 0x7f59ec06ebd0 2024-09-15T03:40:54.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.791+0000 7f5a02ffd700 1 -- 172.21.15.110:0/1216530700 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a0408a440 con 0x7f5a0c141b30 2024-09-15T03:40:54.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.791+0000 7f5a0affd700 1 --2- 172.21.15.110:0/1216530700 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f59ec067a80 0x7f59ec069f30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f59f4006fd0 tx=0x7f59f4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:54.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.791+0000 7f5a0a7fc700 1 --2- 172.21.15.110:0/1216530700 >> [v2:172.21.15.177:6800/22423,v1:172.21.15.177:6801/22423] conn(0x7f59ec06ebd0 0x7f59ec070fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:54.798 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.795+0000 7f5a0a7fc700 1 --2- 172.21.15.110:0/1216530700 >> [v2:172.21.15.177:6800/22423,v1:172.21.15.177:6801/22423] conn(0x7f59ec06ebd0 0x7f59ec070fe0 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:54.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.795+0000 7f5a02ffd700 1 -- 172.21.15.110:0/1216530700 <== osd.7 v2:172.21.15.177:6800/22423 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f59ec071690 con 0x7f59ec06ebd0 2024-09-15T03:40:54.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.815+0000 7f5a1162d700 1 -- 172.21.15.110:0/1216530700 --> [v2:172.21.15.177:6800/22423,v1:172.21.15.177:6801/22423] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f5a0c14dd40 con 0x7f59ec06ebd0 2024-09-15T03:40:54.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.815+0000 7f5a02ffd700 1 -- 172.21.15.110:0/1216530700 <== osd.7 v2:172.21.15.177:6800/22423 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f5a0c14dd40 con 0x7f59ec06ebd0 2024-09-15T03:40:54.818 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-15T03:40:54.818 INFO:teuthology.orchestra.run.smithi110.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T03:40:54.818 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-15T03:40:54.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.815+0000 7f5a1162d700 1 -- 172.21.15.110:0/1216530700 >> [v2:172.21.15.177:6800/22423,v1:172.21.15.177:6801/22423] conn(0x7f59ec06ebd0 msgr2=0x7f59ec070fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:54.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.815+0000 7f5a1162d700 1 --2- 172.21.15.110:0/1216530700 >> [v2:172.21.15.177:6800/22423,v1:172.21.15.177:6801/22423] conn(0x7f59ec06ebd0 0x7f59ec070fe0 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:54.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.815+0000 7f5a1162d700 1 -- 172.21.15.110:0/1216530700 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f59ec067a80 msgr2=0x7f59ec069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:54.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.815+0000 7f5a1162d700 1 --2- 172.21.15.110:0/1216530700 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f59ec067a80 0x7f59ec069f30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f59f4006fd0 tx=0x7f59f4008040 comp rx=0 tx=0).stop 2024-09-15T03:40:54.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.815+0000 7f5a1162d700 1 -- 172.21.15.110:0/1216530700 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a0c141b30 msgr2=0x7f5a0c14d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:54.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.815+0000 7f5a1162d700 1 --2- 172.21.15.110:0/1216530700 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a0c141b30 0x7f5a0c14d3d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c06da50 tx=0x7f5a04019000 comp rx=0 tx=0).stop 2024-09-15T03:40:54.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.815+0000 7f5a0b7fe700 1 -- 172.21.15.110:0/1216530700 reap_dead start 2024-09-15T03:40:54.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.815+0000 7f5a1162d700 1 -- 172.21.15.110:0/1216530700 shutdown_connections 2024-09-15T03:40:54.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.815+0000 7f5a1162d700 1 -- 172.21.15.110:0/1216530700 >> 172.21.15.110:0/1216530700 conn(0x7f5a0c067be0 msgr2=0x7f5a0c0b3d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:40:54.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.815+0000 7f5a1162d700 1 -- 172.21.15.110:0/1216530700 shutdown_connections 2024-09-15T03:40:54.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.815+0000 7f5a1162d700 1 -- 172.21.15.110:0/1216530700 wait complete. 2024-09-15T03:40:54.831 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-15T03:40:54.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.939+0000 7fc8093ed700 1 Processor -- start 2024-09-15T03:40:54.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.939+0000 7fc8093ed700 1 -- start start 2024-09-15T03:40:54.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.939+0000 7fc8093ed700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fc8040ad730 0x7fc80413a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:54.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.939+0000 7fc8093ed700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fc80406cb30 0x7fc8040acb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:54.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.939+0000 7fc8093ed700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc80406c6a0 con 0x7fc8040adb00 2024-09-15T03:40:54.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.939+0000 7fc8093ed700 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8040a4fa0 con 0x7fc80406cb30 2024-09-15T03:40:54.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.939+0000 7fc8093ed700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc80406f2b0 con 0x7fc8040ad730 2024-09-15T03:40:54.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.939+0000 7fc803fff700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fc8040ad730 0x7fc80413a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:54.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.939+0000 7fc808bec700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fc80406cb30 0x7fc8040acb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:54.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.939+0000 7fc803fff700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fc8040ad730 0x7fc80413a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3301/0 says I am v2:172.21.15.110:43316/0 (socket says 172.21.15.110:43316) 2024-09-15T03:40:54.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.939+0000 7fc803fff700 1 -- 172.21.15.110:0/2369174710 learned_addr learned my addr 172.21.15.110:0/2369174710 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:40:54.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.939+0000 7fc803fff700 1 -- 172.21.15.110:0/2369174710 >> v2:172.21.15.110:3300/0 conn(0x7fc80406cb30 msgr2=0x7fc8040acb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:40:54.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.939+0000 7fc803fff700 1 --2- 172.21.15.110:0/2369174710 >> v2:172.21.15.110:3300/0 conn(0x7fc80406cb30 0x7fc8040acb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:54.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.939+0000 7fc803fff700 1 -- 172.21.15.110:0/2369174710 >> v1:172.21.15.177:6789/0 conn(0x7fc8040adb00 legacy=0x7fc80406e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:54.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.939+0000 7fc803fff700 1 -- 172.21.15.110:0/2369174710 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc80406f3f0 con 0x7fc8040ad730 2024-09-15T03:40:54.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.939+0000 7fc808bec700 1 --2- 172.21.15.110:0/2369174710 >> v2:172.21.15.110:3300/0 conn(0x7fc80406cb30 0x7fc8040acb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T03:40:54.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.943+0000 7fc803fff700 1 --2- 172.21.15.110:0/2369174710 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fc8040ad730 0x7fc80413a170 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc7f40099b0 tx=0x7fc7f4009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b055d6a6e5acd3e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:54.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.943+0000 7fc802ffd700 1 -- 172.21.15.110:0/2369174710 <== mon.2 v2:172.21.15.110:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f4009ea0 con 0x7fc8040ad730 2024-09-15T03:40:54.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.943+0000 7fc802ffd700 1 -- 172.21.15.110:0/2369174710 <== mon.2 v2:172.21.15.110:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7f400b860 con 0x7fc8040ad730 2024-09-15T03:40:54.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.943+0000 7fc802ffd700 1 -- 172.21.15.110:0/2369174710 <== mon.2 v2:172.21.15.110:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f400bc00 con 0x7fc8040ad730 2024-09-15T03:40:54.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.943+0000 7fc8093ed700 1 -- 172.21.15.110:0/2369174710 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fc8040ad730 msgr2=0x7fc80413a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:54.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.943+0000 7fc8093ed700 1 --2- 172.21.15.110:0/2369174710 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fc8040ad730 0x7fc80413a170 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc7f40099b0 tx=0x7fc7f4009250 comp rx=0 tx=0).stop 2024-09-15T03:40:54.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.943+0000 7fc8093ed700 1 -- 172.21.15.110:0/2369174710 shutdown_connections 2024-09-15T03:40:54.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.943+0000 7fc8093ed700 1 --2- 172.21.15.110:0/2369174710 >> v2:172.21.15.110:3300/0 conn(0x7fc80406cb30 0x7fc8040acb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:54.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.943+0000 7fc8093ed700 1 --2- 172.21.15.110:0/2369174710 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fc8040ad730 0x7fc80413a170 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:54.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.943+0000 7fc8093ed700 1 -- 172.21.15.110:0/2369174710 >> 172.21.15.110:0/2369174710 conn(0x7fc8040b0cb0 msgr2=0x7fc8040b10b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:40:54.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.943+0000 7fc8093ed700 1 -- 172.21.15.110:0/2369174710 shutdown_connections 2024-09-15T03:40:54.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.943+0000 7fc8093ed700 1 -- 172.21.15.110:0/2369174710 wait complete. 2024-09-15T03:40:54.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.947+0000 7fc8093ed700 1 Processor -- start 2024-09-15T03:40:54.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.947+0000 7fc8093ed700 1 -- start start 2024-09-15T03:40:54.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.947+0000 7fc8093ed700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc80406cb30 0x7fc8040abd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:54.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.947+0000 7fc8093ed700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fc8040ad730 0x7fc80411cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:54.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.947+0000 7fc8093ed700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc8040adb00 0x7fc8040a9140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:54.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.947+0000 7fc8093ed700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc80414da70 con 0x7fc80406cb30 2024-09-15T03:40:54.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.947+0000 7fc8093ed700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc80411d400 con 0x7fc8040adb00 2024-09-15T03:40:54.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.947+0000 7fc8093ed700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8040aaf70 con 0x7fc8040ad730 2024-09-15T03:40:54.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.947+0000 7fc803fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc80406cb30 0x7fc8040abd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:54.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.947+0000 7fc803fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc80406cb30 0x7fc8040abd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51334/0 (socket says 172.21.15.110:51334) 2024-09-15T03:40:54.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.947+0000 7fc803fff700 1 -- 172.21.15.110:0/136264541 learned_addr learned my addr 172.21.15.110:0/136264541 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:40:54.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.947+0000 7fc8037fe700 1 --2- 172.21.15.110:0/136264541 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fc8040ad730 0x7fc80411cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:54.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.947+0000 7fc808bec700 1 --2- 172.21.15.110:0/136264541 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc8040adb00 0x7fc8040a9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:54.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.947+0000 7fc803fff700 1 -- 172.21.15.110:0/136264541 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fc8040ad730 msgr2=0x7fc80411cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:54.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.947+0000 7fc803fff700 1 --2- 172.21.15.110:0/136264541 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fc8040ad730 0x7fc80411cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:54.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.947+0000 7fc803fff700 1 -- 172.21.15.110:0/136264541 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc8040adb00 msgr2=0x7fc8040a9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:54.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.947+0000 7fc803fff700 1 --2- 172.21.15.110:0/136264541 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc8040adb00 0x7fc8040a9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:54.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.947+0000 7fc803fff700 1 -- 172.21.15.110:0/136264541 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7fc00d040 con 0x7fc80406cb30 2024-09-15T03:40:54.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.947+0000 7fc8037fe700 1 --2- 172.21.15.110:0/136264541 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fc8040ad730 0x7fc80411cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T03:40:54.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.947+0000 7fc808bec700 1 --2- 172.21.15.110:0/136264541 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc8040adb00 0x7fc8040a9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T03:40:54.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.947+0000 7fc803fff700 1 --2- 172.21.15.110:0/136264541 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc80406cb30 0x7fc8040abd70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc7f4006fd0 tx=0x7fc7f4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:54.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.947+0000 7fc7ebfff700 1 -- 172.21.15.110:0/136264541 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f4000d10 con 0x7fc80406cb30 2024-09-15T03:40:54.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.947+0000 7fc7ebfff700 1 -- 172.21.15.110:0/136264541 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7f400bbe0 con 0x7fc80406cb30 2024-09-15T03:40:54.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.947+0000 7fc8093ed700 1 -- 172.21.15.110:0/136264541 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7f40094f0 con 0x7fc80406cb30 2024-09-15T03:40:54.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.947+0000 7fc7ebfff700 1 -- 172.21.15.110:0/136264541 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f4011670 con 0x7fc80406cb30 2024-09-15T03:40:54.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.947+0000 7fc8093ed700 1 -- 172.21.15.110:0/136264541 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc80414df00 con 0x7fc80406cb30 2024-09-15T03:40:54.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.951+0000 7fc8093ed700 1 -- 172.21.15.110:0/136264541 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc80414d820 con 0x7fc80406cb30 2024-09-15T03:40:54.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.951+0000 7fc7ebfff700 1 -- 172.21.15.110:0/136264541 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) v1 ==== 85847+0+0 (secure 0 0 0) 0x7fc7f4011aa0 con 0x7fc80406cb30 2024-09-15T03:40:54.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.951+0000 7fc7ebfff700 1 --2- 172.21.15.110:0/136264541 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fc7f0067a80 0x7fc7f0069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:54.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.951+0000 7fc8037fe700 1 --2- 172.21.15.110:0/136264541 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fc7f0067a80 0x7fc7f0069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:54.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.951+0000 7fc7ebfff700 1 -- 172.21.15.110:0/136264541 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5641+0+0 (secure 0 0 0) 0x7fc7f4086e70 con 0x7fc80406cb30 2024-09-15T03:40:54.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.951+0000 7fc8037fe700 1 --2- 172.21.15.110:0/136264541 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fc7f0067a80 0x7fc7f0069f30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc80414d560 tx=0x7fc7f8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:54.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:54.959+0000 7fc7ebfff700 1 -- 172.21.15.110:0/136264541 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fc7f4053fb0 con 0x7fc80406cb30 2024-09-15T03:40:55.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.127+0000 7fc8093ed700 1 -- 172.21.15.110:0/136264541 --> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7fc8040721b0 con 0x7fc7f0067a80 2024-09-15T03:40:55.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.131+0000 7fc7ebfff700 1 -- 172.21.15.110:0/136264541 <== mgr.4104 v2:172.21.15.110:6832/22190 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc8040721b0 con 0x7fc7f0067a80 2024-09-15T03:40:55.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.135+0000 7fc8093ed700 1 -- 172.21.15.110:0/136264541 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fc7f0067a80 msgr2=0x7fc7f0069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:55.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.135+0000 7fc8093ed700 1 --2- 172.21.15.110:0/136264541 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fc7f0067a80 0x7fc7f0069f30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc80414d560 tx=0x7fc7f8009040 comp rx=0 tx=0).stop 2024-09-15T03:40:55.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.135+0000 7fc8093ed700 1 -- 172.21.15.110:0/136264541 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc80406cb30 msgr2=0x7fc8040abd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:55.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.135+0000 7fc8093ed700 1 --2- 172.21.15.110:0/136264541 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc80406cb30 0x7fc8040abd70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc7f4006fd0 tx=0x7fc7f4017000 comp rx=0 tx=0).stop 2024-09-15T03:40:55.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.135+0000 7fc8093ed700 1 -- 172.21.15.110:0/136264541 shutdown_connections 2024-09-15T03:40:55.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.135+0000 7fc8093ed700 1 --2- 172.21.15.110:0/136264541 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fc7f0067a80 0x7fc7f0069f30 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:55.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.135+0000 7fc8093ed700 1 --2- 172.21.15.110:0/136264541 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc8040adb00 0x7fc8040a9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:55.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.135+0000 7fc8093ed700 1 --2- 172.21.15.110:0/136264541 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fc8040ad730 0x7fc80411cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:55.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.135+0000 7fc8093ed700 1 --2- 172.21.15.110:0/136264541 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc80406cb30 0x7fc8040abd70 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:55.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.135+0000 7fc8093ed700 1 -- 172.21.15.110:0/136264541 >> 172.21.15.110:0/136264541 conn(0x7fc8040b0cb0 msgr2=0x7fc80406dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:40:55.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.135+0000 7fc8093ed700 1 -- 172.21.15.110:0/136264541 shutdown_connections 2024-09-15T03:40:55.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.135+0000 7fc8093ed700 1 -- 172.21.15.110:0/136264541 wait complete. 2024-09-15T03:40:55.139 INFO:teuthology.orchestra.run.smithi110.stderr:instructed osd(s) 7 to deep-scrub 2024-09-15T03:40:55.152 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T03:40:55.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.247+0000 7f78f5cba700 1 Processor -- start 2024-09-15T03:40:55.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.247+0000 7f78f5cba700 1 -- start start 2024-09-15T03:40:55.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.251+0000 7f78f5cba700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f78f00adfc0 0x7f78f006e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:55.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.251+0000 7f78f5cba700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f78f013e310 0x7f78f0139bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:55.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.251+0000 7f78f5cba700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78f00ad670 con 0x7f78f00ae390 2024-09-15T03:40:55.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.251+0000 7f78f5cba700 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78f00a50c0 con 0x7f78f013e310 2024-09-15T03:40:55.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.251+0000 7f78f5cba700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78f00a53a0 con 0x7f78f00adfc0 2024-09-15T03:40:55.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.251+0000 7f78ef7fe700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f78f00adfc0 0x7f78f006e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:55.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.251+0000 7f78effff700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f78f013e310 0x7f78f0139bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:55.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.251+0000 7f78effff700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f78f013e310 0x7f78f0139bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:40416/0 (socket says 172.21.15.110:40416) 2024-09-15T03:40:55.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.251+0000 7f78ef7fe700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f78f00adfc0 0x7f78f006e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3301/0 says I am v2:172.21.15.110:43142/0 (socket says 172.21.15.110:43142) 2024-09-15T03:40:55.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.251+0000 7f78effff700 1 -- 172.21.15.110:0/4005678055 learned_addr learned my addr 172.21.15.110:0/4005678055 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:40:55.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.251+0000 7f78ef7fe700 1 -- 172.21.15.110:0/4005678055 >> v2:172.21.15.110:3300/0 conn(0x7f78f013e310 msgr2=0x7f78f0139bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:40:55.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.251+0000 7f78ef7fe700 1 --2- 172.21.15.110:0/4005678055 >> v2:172.21.15.110:3300/0 conn(0x7f78f013e310 0x7f78f0139bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:55.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.251+0000 7f78ef7fe700 1 -- 172.21.15.110:0/4005678055 >> v1:172.21.15.177:6789/0 conn(0x7f78f00ae390 legacy=0x7f78f013a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:55.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.251+0000 7f78ef7fe700 1 -- 172.21.15.110:0/4005678055 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f013ddf0 con 0x7f78f00adfc0 2024-09-15T03:40:55.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.251+0000 7f78effff700 1 --2- 172.21.15.110:0/4005678055 >> v2:172.21.15.110:3300/0 conn(0x7f78f013e310 0x7f78f0139bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T03:40:55.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.251+0000 7f78ef7fe700 1 --2- 172.21.15.110:0/4005678055 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f78f00adfc0 0x7f78f006e780 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f78e00098e0 tx=0x7f78e00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd9fd3d4999d10dd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:55.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.251+0000 7f78eeffd700 1 -- 172.21.15.110:0/4005678055 <== mon.2 v2:172.21.15.110:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e000b640 con 0x7f78f00adfc0 2024-09-15T03:40:55.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.251+0000 7f78eeffd700 1 -- 172.21.15.110:0/4005678055 <== mon.2 v2:172.21.15.110:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78e000b7a0 con 0x7f78f00adfc0 2024-09-15T03:40:55.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.251+0000 7f78eeffd700 1 -- 172.21.15.110:0/4005678055 <== mon.2 v2:172.21.15.110:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e000bb40 con 0x7f78f00adfc0 2024-09-15T03:40:55.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.255+0000 7f78f5cba700 1 -- 172.21.15.110:0/4005678055 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f78f00adfc0 msgr2=0x7f78f006e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:55.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.255+0000 7f78f5cba700 1 --2- 172.21.15.110:0/4005678055 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f78f00adfc0 0x7f78f006e780 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f78e00098e0 tx=0x7f78e00091a0 comp rx=0 tx=0).stop 2024-09-15T03:40:55.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.255+0000 7f78f5cba700 1 -- 172.21.15.110:0/4005678055 shutdown_connections 2024-09-15T03:40:55.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.255+0000 7f78f5cba700 1 --2- 172.21.15.110:0/4005678055 >> v2:172.21.15.110:3300/0 conn(0x7f78f013e310 0x7f78f0139bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:55.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.255+0000 7f78f5cba700 1 --2- 172.21.15.110:0/4005678055 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f78f00adfc0 0x7f78f006e780 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:55.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.255+0000 7f78f5cba700 1 -- 172.21.15.110:0/4005678055 >> 172.21.15.110:0/4005678055 conn(0x7f78f00b1840 msgr2=0x7f78f00ad8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:40:55.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.255+0000 7f78f5cba700 1 -- 172.21.15.110:0/4005678055 shutdown_connections 2024-09-15T03:40:55.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.255+0000 7f78f5cba700 1 -- 172.21.15.110:0/4005678055 wait complete. 2024-09-15T03:40:55.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.255+0000 7f78f5cba700 1 Processor -- start 2024-09-15T03:40:55.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.255+0000 7f78f5cba700 1 -- start start 2024-09-15T03:40:55.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.259+0000 7f78f5cba700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f78f00adfc0 0x7f78f014f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:55.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.259+0000 7f78f5cba700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78f00ae390 0x7f78f0137c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:55.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.259+0000 7f78f5cba700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f78f013e310 0x7f78f01388c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:55.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.259+0000 7f78f5cba700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f01383a0 con 0x7f78f00ae390 2024-09-15T03:40:55.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.259+0000 7f78f5cba700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f014cba0 con 0x7f78f00adfc0 2024-09-15T03:40:55.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.259+0000 7f78f5cba700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78f014fd40 con 0x7f78f013e310 2024-09-15T03:40:55.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.259+0000 7f78effff700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f78f013e310 0x7f78f01388c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:55.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.259+0000 7f78e7fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78f00ae390 0x7f78f0137c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:55.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.259+0000 7f78effff700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f78f013e310 0x7f78f01388c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3301/0 says I am v2:172.21.15.110:43158/0 (socket says 172.21.15.110:43158) 2024-09-15T03:40:55.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.259+0000 7f78effff700 1 -- 172.21.15.110:0/99193322 learned_addr learned my addr 172.21.15.110:0/99193322 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:40:55.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.259+0000 7f78ef7fe700 1 --2- 172.21.15.110:0/99193322 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f78f00adfc0 0x7f78f014f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:55.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.259+0000 7f78effff700 1 -- 172.21.15.110:0/99193322 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f78f00adfc0 msgr2=0x7f78f014f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:55.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.259+0000 7f78effff700 1 --2- 172.21.15.110:0/99193322 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f78f00adfc0 0x7f78f014f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:55.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.259+0000 7f78effff700 1 -- 172.21.15.110:0/99193322 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78f00ae390 msgr2=0x7f78f0137c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:55.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.259+0000 7f78effff700 1 --2- 172.21.15.110:0/99193322 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78f00ae390 0x7f78f0137c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:55.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.259+0000 7f78effff700 1 -- 172.21.15.110:0/99193322 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78e8009ae0 con 0x7f78f013e310 2024-09-15T03:40:55.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.259+0000 7f78effff700 1 --2- 172.21.15.110:0/99193322 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f78f013e310 0x7f78f01388c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f78e8009f30 tx=0x7f78e800b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:55.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.259+0000 7f78e77fe700 1 -- 172.21.15.110:0/99193322 <== mon.2 v2:172.21.15.110:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e8018070 con 0x7f78f013e310 2024-09-15T03:40:55.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.259+0000 7f78e77fe700 1 -- 172.21.15.110:0/99193322 <== mon.2 v2:172.21.15.110:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78e800f040 con 0x7f78f013e310 2024-09-15T03:40:55.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.259+0000 7f78f5cba700 1 -- 172.21.15.110:0/99193322 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78e0009420 con 0x7f78f013e310 2024-09-15T03:40:55.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.259+0000 7f78e77fe700 1 -- 172.21.15.110:0/99193322 <== mon.2 v2:172.21.15.110:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e800cb70 con 0x7f78f013e310 2024-09-15T03:40:55.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.259+0000 7f78f5cba700 1 -- 172.21.15.110:0/99193322 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78f0135860 con 0x7f78f013e310 2024-09-15T03:40:55.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.259+0000 7f78f5cba700 1 -- 172.21.15.110:0/99193322 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78f00a4820 con 0x7f78f013e310 2024-09-15T03:40:55.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.259+0000 7f78e77fe700 1 -- 172.21.15.110:0/99193322 <== mon.2 v2:172.21.15.110:3301/0 4 ==== mgrmap(e 5) v1 ==== 85847+0+0 (secure 0 0 0) 0x7f78e8003020 con 0x7f78f013e310 2024-09-15T03:40:55.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.263+0000 7f78e77fe700 1 --2- 172.21.15.110:0/99193322 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f78d0067a80 0x7f78d0069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:40:55.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.263+0000 7f78e77fe700 1 -- 172.21.15.110:0/99193322 <== mon.2 v2:172.21.15.110:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5641+0+0 (secure 0 0 0) 0x7f78e80855f0 con 0x7f78f013e310 2024-09-15T03:40:55.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.263+0000 7f78ef7fe700 1 --2- 172.21.15.110:0/99193322 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f78d0067a80 0x7f78d0069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:40:55.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.263+0000 7f78ef7fe700 1 --2- 172.21.15.110:0/99193322 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f78d0067a80 0x7f78d0069f30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f78e00093f0 tx=0x7f78e0017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:40:55.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.271+0000 7f78e77fe700 1 -- 172.21.15.110:0/99193322 <== mon.2 v2:172.21.15.110:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f78f00a4820 con 0x7f78f013e310 2024-09-15T03:40:55.436 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.431+0000 7f78f5cba700 1 -- 172.21.15.110:0/99193322 --> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f78f00a4820 con 0x7f78d0067a80 2024-09-15T03:40:55.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.443+0000 7f78e77fe700 1 -- 172.21.15.110:0/99193322 <== mgr.4104 v2:172.21.15.110:6832/22190 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+153246 (secure 0 0 0) 0x7f78f00a4820 con 0x7f78d0067a80 2024-09-15T03:40:55.451 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-15T03:40:55.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.447+0000 7f78f5cba700 1 -- 172.21.15.110:0/99193322 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f78d0067a80 msgr2=0x7f78d0069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:55.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.447+0000 7f78f5cba700 1 --2- 172.21.15.110:0/99193322 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f78d0067a80 0x7f78d0069f30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f78e00093f0 tx=0x7f78e0017000 comp rx=0 tx=0).stop 2024-09-15T03:40:55.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.447+0000 7f78f5cba700 1 -- 172.21.15.110:0/99193322 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f78f013e310 msgr2=0x7f78f01388c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:40:55.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.447+0000 7f78f5cba700 1 --2- 172.21.15.110:0/99193322 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f78f013e310 0x7f78f01388c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f78e8009f30 tx=0x7f78e800b040 comp rx=0 tx=0).stop 2024-09-15T03:40:55.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.447+0000 7f78f5cba700 1 -- 172.21.15.110:0/99193322 shutdown_connections 2024-09-15T03:40:55.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.447+0000 7f78f5cba700 1 --2- 172.21.15.110:0/99193322 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f78d0067a80 0x7f78d0069f30 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:55.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.447+0000 7f78f5cba700 1 --2- 172.21.15.110:0/99193322 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f78f013e310 0x7f78f01388c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:55.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.447+0000 7f78f5cba700 1 --2- 172.21.15.110:0/99193322 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78f00ae390 0x7f78f0137c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:55.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.447+0000 7f78f5cba700 1 --2- 172.21.15.110:0/99193322 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f78f00adfc0 0x7f78f014f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:40:55.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.447+0000 7f78f5cba700 1 -- 172.21.15.110:0/99193322 >> 172.21.15.110:0/99193322 conn(0x7f78f00b1840 msgr2=0x7f78f013ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:40:55.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.447+0000 7f78f5cba700 1 -- 172.21.15.110:0/99193322 shutdown_connections 2024-09-15T03:40:55.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:40:55.447+0000 7f78f5cba700 1 -- 172.21.15.110:0/99193322 wait complete. 2024-09-15T03:40:55.456 INFO:teuthology.orchestra.run.smithi110.stderr:dumped all 2024-09-15T03:40:55.467 INFO:teuthology.orchestra.run.smithi110.stdout:{"pg_ready":true,"pg_map":{"version":146,"stamp":"2024-09-15T03:40:54.678245+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3570754347,"num_objects":882,"num_object_clones":0,"num_object_copies":1764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":882,"num_whiteouts":0,"num_read":2629,"num_read_kb":2634,"num_write":159122,"num_write_kb":5635367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160867,"ondisk_log_size":160867,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":9118228,"kb_used_data":6685304,"kb_used_omap":13,"kb_used_meta":2432818,"kb_avail":745856492,"statfs":{"total":773094113280,"available":763757047808,"internally_reserved":0,"allocated":6845751296,"data_stored":6840210502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2491206336},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8720,"num_objects":-1,"num_object_clones":0,"num_object_copies":-2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1,"num_whiteouts":0,"num_read":66,"num_read_kb":71,"num_write":17,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003166"},"pg_stats":[{"pgid":"3.1f","version":"22'3025","reported_seq":3036,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:32.398466+0000","last_change":"2024-09-15T03:36:48.151935+0000","last_active":"2024-09-15T03:39:32.398466+0000","last_peered":"2024-09-15T03:39:32.398466+0000","last_clean":"2024-09-15T03:39:32.398466+0000","last_became_active":"2024-09-15T03:36:48.151625+0000","last_became_peered":"2024-09-15T03:36:48.151625+0000","last_unstale":"2024-09-15T03:39:32.398466+0000","last_undegraded":"2024-09-15T03:39:32.398466+0000","last_fullsized":"2024-09-15T03:39:32.398466+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":3025,"log_dups_size":0,"ondisk_log_size":3025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:02:31.103276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663296,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3019,"num_write_kb":136067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"24'6208","reported_seq":6525,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:48.328565+0000","last_change":"2024-09-15T03:36:48.150055+0000","last_active":"2024-09-15T03:40:48.328565+0000","last_peered":"2024-09-15T03:40:48.328565+0000","last_clean":"2024-09-15T03:40:48.328565+0000","last_became_active":"2024-09-15T03:36:48.149731+0000","last_became_peered":"2024-09-15T03:36:48.149731+0000","last_unstale":"2024-09-15T03:40:48.328565+0000","last_undegraded":"2024-09-15T03:40:48.328565+0000","last_fullsized":"2024-09-15T03:40:48.328565+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":6208,"log_dups_size":0,"ondisk_log_size":6208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113377280,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":1022,"num_read_kb":916,"num_write":6394,"num_write_kb":178051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"24'6288","reported_seq":6402,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:48.322235+0000","last_change":"2024-09-15T03:36:48.162431+0000","last_active":"2024-09-15T03:40:48.322235+0000","last_peered":"2024-09-15T03:40:48.322235+0000","last_clean":"2024-09-15T03:40:48.322235+0000","last_became_active":"2024-09-15T03:36:48.162085+0000","last_became_peered":"2024-09-15T03:36:48.162085+0000","last_unstale":"2024-09-15T03:40:48.322235+0000","last_undegraded":"2024-09-15T03:40:48.322235+0000","last_fullsized":"2024-09-15T03:40:48.322235+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":6288,"log_dups_size":0,"ondisk_log_size":6288,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:51:54.108247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121898016,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":109,"num_read_kb":107,"num_write":6275,"num_write_kb":177699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"22'4656","reported_seq":4675,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:32.448301+0000","last_change":"2024-09-15T03:36:48.159640+0000","last_active":"2024-09-15T03:39:32.448301+0000","last_peered":"2024-09-15T03:39:32.448301+0000","last_clean":"2024-09-15T03:39:32.448301+0000","last_became_active":"2024-09-15T03:36:48.159272+0000","last_became_peered":"2024-09-15T03:36:48.159272+0000","last_unstale":"2024-09-15T03:39:32.448301+0000","last_undegraded":"2024-09-15T03:39:32.448301+0000","last_fullsized":"2024-09-15T03:39:32.448301+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":4656,"log_dups_size":0,"ondisk_log_size":4656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:17:25.869833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88866816,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":4665,"num_write_kb":169740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"24'4798","reported_seq":4809,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:48.325892+0000","last_change":"2024-09-15T03:36:48.150332+0000","last_active":"2024-09-15T03:40:48.325892+0000","last_peered":"2024-09-15T03:40:48.325892+0000","last_clean":"2024-09-15T03:40:48.325892+0000","last_became_active":"2024-09-15T03:36:48.150057+0000","last_became_peered":"2024-09-15T03:36:48.150057+0000","last_unstale":"2024-09-15T03:40:48.325892+0000","last_undegraded":"2024-09-15T03:40:48.325892+0000","last_fullsized":"2024-09-15T03:40:48.325892+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":4798,"log_dups_size":0,"ondisk_log_size":4798,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:15:39.747133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117440512,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4750,"num_write_kb":185221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"24'4714","reported_seq":4730,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:48.265954+0000","last_change":"2024-09-15T03:36:48.144892+0000","last_active":"2024-09-15T03:40:48.265954+0000","last_peered":"2024-09-15T03:40:48.265954+0000","last_clean":"2024-09-15T03:40:48.265954+0000","last_became_active":"2024-09-15T03:36:48.144097+0000","last_became_peered":"2024-09-15T03:36:48.144097+0000","last_unstale":"2024-09-15T03:40:48.265954+0000","last_undegraded":"2024-09-15T03:40:48.265954+0000","last_fullsized":"2024-09-15T03:40:48.265954+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":4714,"log_dups_size":0,"ondisk_log_size":4714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:16:01.325609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68681728,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":22,"num_read_kb":16,"num_write":4699,"num_write_kb":143618,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"24'4493","reported_seq":4505,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:48.324914+0000","last_change":"2024-09-15T03:36:48.142399+0000","last_active":"2024-09-15T03:40:48.324914+0000","last_peered":"2024-09-15T03:40:48.324914+0000","last_clean":"2024-09-15T03:40:48.324914+0000","last_became_active":"2024-09-15T03:36:48.142085+0000","last_became_peered":"2024-09-15T03:36:48.142085+0000","last_unstale":"2024-09-15T03:40:48.324914+0000","last_undegraded":"2024-09-15T03:40:48.324914+0000","last_fullsized":"2024-09-15T03:40:48.324914+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":4493,"log_dups_size":0,"ondisk_log_size":4493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T13:00:10.524261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":111872344,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":4448,"num_write_kb":183583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"23'7205","reported_seq":7221,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:48.252708+0000","last_change":"2024-09-15T03:36:48.138540+0000","last_active":"2024-09-15T03:40:48.252708+0000","last_peered":"2024-09-15T03:40:48.252708+0000","last_clean":"2024-09-15T03:40:48.252708+0000","last_became_active":"2024-09-15T03:36:48.138049+0000","last_became_peered":"2024-09-15T03:36:48.138049+0000","last_unstale":"2024-09-15T03:40:48.252708+0000","last_undegraded":"2024-09-15T03:40:48.252708+0000","last_fullsized":"2024-09-15T03:40:48.252708+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":7205,"log_dups_size":0,"ondisk_log_size":7205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113246720,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":15,"num_read_kb":11,"num_write":7218,"num_write_kb":193736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"23'7862","reported_seq":7880,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:54.844917+0000","last_change":"2024-09-15T03:36:48.150100+0000","last_active":"2024-09-15T03:39:54.844917+0000","last_peered":"2024-09-15T03:39:54.844917+0000","last_clean":"2024-09-15T03:39:54.844917+0000","last_became_active":"2024-09-15T03:36:48.149792+0000","last_became_peered":"2024-09-15T03:36:48.149792+0000","last_unstale":"2024-09-15T03:39:54.844917+0000","last_undegraded":"2024-09-15T03:39:54.844917+0000","last_fullsized":"2024-09-15T03:39:54.844917+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":7862,"log_dups_size":0,"ondisk_log_size":7862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:42:07.633298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":146800640,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":7874,"num_write_kb":194640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"24'2409","reported_seq":2423,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:48.320093+0000","last_change":"2024-09-15T03:36:48.155132+0000","last_active":"2024-09-15T03:40:48.320093+0000","last_peered":"2024-09-15T03:40:48.320093+0000","last_clean":"2024-09-15T03:40:48.320093+0000","last_became_active":"2024-09-15T03:36:48.154800+0000","last_became_peered":"2024-09-15T03:36:48.154800+0000","last_unstale":"2024-09-15T03:40:48.320093+0000","last_undegraded":"2024-09-15T03:40:48.320093+0000","last_fullsized":"2024-09-15T03:40:48.320093+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":2409,"log_dups_size":0,"ondisk_log_size":2409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:54:21.705564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":2388,"num_write_kb":127362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"23'6301","reported_seq":6318,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:48.735332+0000","last_change":"2024-09-15T03:36:48.152499+0000","last_active":"2024-09-15T03:39:48.735332+0000","last_peered":"2024-09-15T03:39:48.735332+0000","last_clean":"2024-09-15T03:39:48.735332+0000","last_became_active":"2024-09-15T03:36:48.152191+0000","last_became_peered":"2024-09-15T03:36:48.152191+0000","last_unstale":"2024-09-15T03:39:48.735332+0000","last_undegraded":"2024-09-15T03:39:48.735332+0000","last_fullsized":"2024-09-15T03:39:48.735332+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":6301,"log_dups_size":0,"ondisk_log_size":6301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T12:12:56.810421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":124518928,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":19,"num_read_kb":14,"num_write":6316,"num_write_kb":168780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"24'5387","reported_seq":5401,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:48.331319+0000","last_change":"2024-09-15T03:36:48.154566+0000","last_active":"2024-09-15T03:40:48.331319+0000","last_peered":"2024-09-15T03:40:48.331319+0000","last_clean":"2024-09-15T03:40:48.331319+0000","last_became_active":"2024-09-15T03:36:48.153926+0000","last_became_peered":"2024-09-15T03:36:48.153926+0000","last_unstale":"2024-09-15T03:40:48.331319+0000","last_undegraded":"2024-09-15T03:40:48.331319+0000","last_fullsized":"2024-09-15T03:40:48.331319+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":5387,"log_dups_size":0,"ondisk_log_size":5387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:19:06.740804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109121016,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":5342,"num_write_kb":144586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:32.448890+0000","last_change":"2024-09-15T03:36:42.814645+0000","last_active":"2024-09-15T03:39:32.448890+0000","last_peered":"2024-09-15T03:39:32.448890+0000","last_clean":"2024-09-15T03:39:32.448890+0000","last_became_active":"2024-09-15T03:36:40.672828+0000","last_became_peered":"2024-09-15T03:36:40.672828+0000","last_unstale":"2024-09-15T03:39:32.448890+0000","last_undegraded":"2024-09-15T03:39:32.448890+0000","last_fullsized":"2024-09-15T03:39:32.448890+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_clean_scrub_stamp":"2024-09-15T03:36:39.639431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T13:40:53.039569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064069099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"24'5982","reported_seq":6016,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:48.254676+0000","last_change":"2024-09-15T03:36:48.143463+0000","last_active":"2024-09-15T03:40:48.254676+0000","last_peered":"2024-09-15T03:40:48.254676+0000","last_clean":"2024-09-15T03:40:48.254676+0000","last_became_active":"2024-09-15T03:36:48.143177+0000","last_became_peered":"2024-09-15T03:36:48.143177+0000","last_unstale":"2024-09-15T03:40:48.254676+0000","last_undegraded":"2024-09-15T03:40:48.254676+0000","last_fullsized":"2024-09-15T03:40:48.254676+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":5982,"log_dups_size":0,"ondisk_log_size":5982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:20:24.926833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":134639616,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":92,"num_read_kb":80,"num_write":5989,"num_write_kb":192351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:32.447432+0000","last_change":"2024-09-15T03:36:42.687267+0000","last_active":"2024-09-15T03:39:32.447432+0000","last_peered":"2024-09-15T03:39:32.447432+0000","last_clean":"2024-09-15T03:39:32.447432+0000","last_became_active":"2024-09-15T03:36:40.671677+0000","last_became_peered":"2024-09-15T03:36:40.671677+0000","last_unstale":"2024-09-15T03:39:32.447432+0000","last_undegraded":"2024-09-15T03:39:32.447432+0000","last_fullsized":"2024-09-15T03:39:32.447432+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_clean_scrub_stamp":"2024-09-15T03:36:39.639431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:55:15.493964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000861089,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"24'5402","reported_seq":5414,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:48.337236+0000","last_change":"2024-09-15T03:36:48.152124+0000","last_active":"2024-09-15T03:40:48.337236+0000","last_peered":"2024-09-15T03:40:48.337236+0000","last_clean":"2024-09-15T03:40:48.337236+0000","last_became_active":"2024-09-15T03:36:48.151844+0000","last_became_peered":"2024-09-15T03:36:48.151844+0000","last_unstale":"2024-09-15T03:40:48.337236+0000","last_undegraded":"2024-09-15T03:40:48.337236+0000","last_fullsized":"2024-09-15T03:40:48.337236+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":5402,"log_dups_size":0,"ondisk_log_size":5402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":115998720,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5352,"num_write_kb":174984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:32.411245+0000","last_change":"2024-09-15T03:36:42.854270+0000","last_active":"2024-09-15T03:39:32.411245+0000","last_peered":"2024-09-15T03:39:32.411245+0000","last_clean":"2024-09-15T03:39:32.411245+0000","last_became_active":"2024-09-15T03:36:41.079216+0000","last_became_peered":"2024-09-15T03:36:41.079216+0000","last_unstale":"2024-09-15T03:39:32.411245+0000","last_undegraded":"2024-09-15T03:39:32.411245+0000","last_fullsized":"2024-09-15T03:39:32.411245+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_clean_scrub_stamp":"2024-09-15T03:36:39.639431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:19:37.610607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062568200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"23'7270","reported_seq":7292,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:53.196102+0000","last_change":"2024-09-15T03:36:48.216882+0000","last_active":"2024-09-15T03:39:53.196102+0000","last_peered":"2024-09-15T03:39:53.196102+0000","last_clean":"2024-09-15T03:39:53.196102+0000","last_became_active":"2024-09-15T03:36:48.216337+0000","last_became_peered":"2024-09-15T03:36:48.216337+0000","last_unstale":"2024-09-15T03:39:53.196102+0000","last_undegraded":"2024-09-15T03:39:53.196102+0000","last_fullsized":"2024-09-15T03:39:53.196102+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":7270,"log_dups_size":0,"ondisk_log_size":7270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:27:25.000492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":146931712,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":45,"num_read_kb":34,"num_write":7284,"num_write_kb":246402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:32.448012+0000","last_change":"2024-09-15T03:36:42.687270+0000","last_active":"2024-09-15T03:39:32.448012+0000","last_peered":"2024-09-15T03:39:32.448012+0000","last_clean":"2024-09-15T03:39:32.448012+0000","last_became_active":"2024-09-15T03:36:41.077266+0000","last_became_peered":"2024-09-15T03:36:41.077266+0000","last_unstale":"2024-09-15T03:39:32.448012+0000","last_undegraded":"2024-09-15T03:39:32.448012+0000","last_fullsized":"2024-09-15T03:39:32.448012+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_clean_scrub_stamp":"2024-09-15T03:36:39.639431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:54:58.435053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067514400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"22'3282","reported_seq":3293,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:32.448767+0000","last_change":"2024-09-15T03:36:48.151841+0000","last_active":"2024-09-15T03:39:32.448767+0000","last_peered":"2024-09-15T03:39:32.448767+0000","last_clean":"2024-09-15T03:39:32.448767+0000","last_became_active":"2024-09-15T03:36:48.151520+0000","last_became_peered":"2024-09-15T03:36:48.151520+0000","last_unstale":"2024-09-15T03:39:32.448767+0000","last_undegraded":"2024-09-15T03:39:32.448767+0000","last_fullsized":"2024-09-15T03:39:32.448767+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":3282,"log_dups_size":0,"ondisk_log_size":3282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:24:47.104486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3278,"num_write_kb":144390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"22'3449","reported_seq":3474,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:32.395623+0000","last_change":"2024-09-15T03:36:48.146454+0000","last_active":"2024-09-15T03:39:32.395623+0000","last_peered":"2024-09-15T03:39:32.395623+0000","last_clean":"2024-09-15T03:39:32.395623+0000","last_became_active":"2024-09-15T03:36:48.145845+0000","last_became_peered":"2024-09-15T03:36:48.145845+0000","last_unstale":"2024-09-15T03:39:32.395623+0000","last_undegraded":"2024-09-15T03:39:32.395623+0000","last_fullsized":"2024-09-15T03:39:32.395623+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":3449,"log_dups_size":0,"ondisk_log_size":3449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:46:57.211180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113246208,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":50,"num_read_kb":41,"num_write":3479,"num_write_kb":144783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:32.448183+0000","last_change":"2024-09-15T03:36:42.695885+0000","last_active":"2024-09-15T03:39:32.448183+0000","last_peered":"2024-09-15T03:39:32.448183+0000","last_clean":"2024-09-15T03:39:32.448183+0000","last_became_active":"2024-09-15T03:36:40.673743+0000","last_became_peered":"2024-09-15T03:36:40.673743+0000","last_unstale":"2024-09-15T03:39:32.448183+0000","last_undegraded":"2024-09-15T03:39:32.448183+0000","last_fullsized":"2024-09-15T03:39:32.448183+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_clean_scrub_stamp":"2024-09-15T03:36:39.639431+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T13:57:31.462981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002159518,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"24'4712","reported_seq":4737,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:48.327813+0000","last_change":"2024-09-15T03:36:48.216887+0000","last_active":"2024-09-15T03:40:48.327813+0000","last_peered":"2024-09-15T03:40:48.327813+0000","last_clean":"2024-09-15T03:40:48.327813+0000","last_became_active":"2024-09-15T03:36:48.216269+0000","last_became_peered":"2024-09-15T03:36:48.216269+0000","last_unstale":"2024-09-15T03:40:48.327813+0000","last_undegraded":"2024-09-15T03:40:48.327813+0000","last_fullsized":"2024-09-15T03:40:48.327813+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":4712,"log_dups_size":0,"ondisk_log_size":4712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T12:10:11.804094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":142877200,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":51,"num_read_kb":164,"num_write":4698,"num_write_kb":182422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":31,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:53.651919+0000","last_change":"2024-09-15T03:40:53.651875+0000","last_active":"2024-09-15T03:40:53.651919+0000","last_peered":"2024-09-15T03:40:53.651919+0000","last_clean":"2024-09-15T03:40:53.651919+0000","last_became_active":"2024-09-15T03:36:40.669392+0000","last_became_peered":"2024-09-15T03:36:40.669392+0000","last_unstale":"2024-09-15T03:40:53.651919+0000","last_undegraded":"2024-09-15T03:40:53.651919+0000","last_fullsized":"2024-09-15T03:40:53.651919+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:40:53.651715+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:40:53.651715+0000","last_clean_scrub_stamp":"2024-09-15T03:40:53.651715+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:08:35.720046+0000","scrub_duration":0.0021291890000000001,"objects_trimmed":0,"snaptrim_duration":0.00059629799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"24'3228","reported_seq":3247,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:48.254290+0000","last_change":"2024-09-15T03:36:48.144585+0000","last_active":"2024-09-15T03:40:48.254290+0000","last_peered":"2024-09-15T03:40:48.254290+0000","last_clean":"2024-09-15T03:40:48.254290+0000","last_became_active":"2024-09-15T03:36:48.144313+0000","last_became_peered":"2024-09-15T03:36:48.144313+0000","last_unstale":"2024-09-15T03:40:48.254290+0000","last_undegraded":"2024-09-15T03:40:48.254290+0000","last_fullsized":"2024-09-15T03:40:48.254290+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":3228,"log_dups_size":0,"ondisk_log_size":3228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T15:27:39.715172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92405760,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":33,"num_read_kb":87,"num_write":3327,"num_write_kb":128260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:32.410323+0000","last_change":"2024-09-15T03:36:42.854222+0000","last_active":"2024-09-15T03:39:32.410323+0000","last_peered":"2024-09-15T03:39:32.410323+0000","last_clean":"2024-09-15T03:39:32.410323+0000","last_became_active":"2024-09-15T03:36:40.672416+0000","last_became_peered":"2024-09-15T03:36:40.672416+0000","last_unstale":"2024-09-15T03:39:32.410323+0000","last_undegraded":"2024-09-15T03:39:32.410323+0000","last_fullsized":"2024-09-15T03:39:32.410323+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_clean_scrub_stamp":"2024-09-15T03:36:39.639431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T15:08:21.420921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065187600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"24'3123","reported_seq":3145,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:53.585199+0000","last_change":"2024-09-15T03:40:53.585158+0000","last_active":"2024-09-15T03:40:53.585199+0000","last_peered":"2024-09-15T03:40:53.585199+0000","last_clean":"2024-09-15T03:40:53.585199+0000","last_became_active":"2024-09-15T03:36:48.150693+0000","last_became_peered":"2024-09-15T03:36:48.150693+0000","last_unstale":"2024-09-15T03:40:53.585199+0000","last_undegraded":"2024-09-15T03:40:53.585199+0000","last_fullsized":"2024-09-15T03:40:53.585199+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'3123","last_scrub_stamp":"2024-09-15T03:40:53.585070+0000","last_deep_scrub":"24'3123","last_deep_scrub_stamp":"2024-09-15T03:40:53.585070+0000","last_clean_scrub_stamp":"2024-09-15T03:40:53.585070+0000","objects_scrubbed":16,"log_size":3123,"log_dups_size":0,"ondisk_log_size":3123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:13:52.170594+0000","scrub_duration":0.174875789,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67108864,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3105,"num_write_kb":103402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:32.448121+0000","last_change":"2024-09-15T03:36:42.694486+0000","last_active":"2024-09-15T03:39:32.448121+0000","last_peered":"2024-09-15T03:39:32.448121+0000","last_clean":"2024-09-15T03:39:32.448121+0000","last_became_active":"2024-09-15T03:36:40.674442+0000","last_became_peered":"2024-09-15T03:36:40.674442+0000","last_unstale":"2024-09-15T03:39:32.448121+0000","last_undegraded":"2024-09-15T03:39:32.448121+0000","last_fullsized":"2024-09-15T03:39:32.448121+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:39.639431+0000","last_clean_scrub_stamp":"2024-09-15T03:36:39.639431+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:25:57.474430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067001500000000004,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":107,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:32.410487+0000","last_change":"2024-09-15T03:36:38.815652+0000","last_active":"2024-09-15T03:39:32.410487+0000","last_peered":"2024-09-15T03:39:32.410487+0000","last_clean":"2024-09-15T03:39:32.410487+0000","last_became_active":"2024-09-15T03:36:38.813490+0000","last_became_peered":"2024-09-15T03:36:38.813490+0000","last_unstale":"2024-09-15T03:39:32.410487+0000","last_undegraded":"2024-09-15T03:39:32.410487+0000","last_fullsized":"2024-09-15T03:39:32.410487+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:37.618040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:37.618040+0000","last_clean_scrub_stamp":"2024-09-15T03:36:37.618040+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:02:12.095255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"24'5190","reported_seq":5206,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:48.329122+0000","last_change":"2024-09-15T03:36:48.142652+0000","last_active":"2024-09-15T03:40:48.329122+0000","last_peered":"2024-09-15T03:40:48.329122+0000","last_clean":"2024-09-15T03:40:48.329122+0000","last_became_active":"2024-09-15T03:36:48.142313+0000","last_became_peered":"2024-09-15T03:36:48.142313+0000","last_unstale":"2024-09-15T03:40:48.329122+0000","last_undegraded":"2024-09-15T03:40:48.329122+0000","last_fullsized":"2024-09-15T03:40:48.329122+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":5190,"log_dups_size":0,"ondisk_log_size":5190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":116760576,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":12,"num_read_kb":135,"num_write":5130,"num_write_kb":177200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"24'7945","reported_seq":8071,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:48.359030+0000","last_change":"2024-09-15T03:36:48.218591+0000","last_active":"2024-09-15T03:40:48.359030+0000","last_peered":"2024-09-15T03:40:48.359030+0000","last_clean":"2024-09-15T03:40:48.359030+0000","last_became_active":"2024-09-15T03:36:48.218083+0000","last_became_peered":"2024-09-15T03:36:48.218083+0000","last_unstale":"2024-09-15T03:40:48.359030+0000","last_undegraded":"2024-09-15T03:40:48.359030+0000","last_fullsized":"2024-09-15T03:40:48.359030+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":7945,"log_dups_size":0,"ondisk_log_size":7945,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:15:32.571275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663808,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":381,"num_read_kb":409,"num_write":5985,"num_write_kb":152253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"24'5837","reported_seq":5868,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:48.324243+0000","last_change":"2024-09-15T03:36:48.155747+0000","last_active":"2024-09-15T03:40:48.324243+0000","last_peered":"2024-09-15T03:40:48.324243+0000","last_clean":"2024-09-15T03:40:48.324243+0000","last_became_active":"2024-09-15T03:36:48.155263+0000","last_became_peered":"2024-09-15T03:36:48.155263+0000","last_unstale":"2024-09-15T03:40:48.324243+0000","last_undegraded":"2024-09-15T03:40:48.324243+0000","last_fullsized":"2024-09-15T03:40:48.324243+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":5837,"log_dups_size":0,"ondisk_log_size":5837,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:40:42.479597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":137103712,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":68,"num_read_kb":57,"num_write":5841,"num_write_kb":205281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"22'3150","reported_seq":3162,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:48.252880+0000","last_change":"2024-09-15T03:36:48.138000+0000","last_active":"2024-09-15T03:40:48.252880+0000","last_peered":"2024-09-15T03:40:48.252880+0000","last_clean":"2024-09-15T03:40:48.252880+0000","last_became_active":"2024-09-15T03:36:48.137589+0000","last_became_peered":"2024-09-15T03:36:48.137589+0000","last_unstale":"2024-09-15T03:40:48.252880+0000","last_undegraded":"2024-09-15T03:40:48.252880+0000","last_fullsized":"2024-09-15T03:40:48.252880+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":3150,"log_dups_size":0,"ondisk_log_size":3150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3149,"num_write_kb":152324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"24'4732","reported_seq":4760,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:48.330119+0000","last_change":"2024-09-15T03:36:48.151666+0000","last_active":"2024-09-15T03:40:48.330119+0000","last_peered":"2024-09-15T03:40:48.330119+0000","last_clean":"2024-09-15T03:40:48.330119+0000","last_became_active":"2024-09-15T03:36:48.151365+0000","last_became_peered":"2024-09-15T03:36:48.151365+0000","last_unstale":"2024-09-15T03:40:48.330119+0000","last_undegraded":"2024-09-15T03:40:48.330119+0000","last_fullsized":"2024-09-15T03:40:48.330119+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":4732,"log_dups_size":0,"ondisk_log_size":4732,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T15:05:31.496486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121635328,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":72,"num_read_kb":60,"num_write":4709,"num_write_kb":215880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"24'5518","reported_seq":5533,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:48.256486+0000","last_change":"2024-09-15T03:36:48.155332+0000","last_active":"2024-09-15T03:40:48.256486+0000","last_peered":"2024-09-15T03:40:48.256486+0000","last_clean":"2024-09-15T03:40:48.256486+0000","last_became_active":"2024-09-15T03:36:48.154955+0000","last_became_peered":"2024-09-15T03:36:48.154955+0000","last_unstale":"2024-09-15T03:40:48.256486+0000","last_undegraded":"2024-09-15T03:40:48.256486+0000","last_fullsized":"2024-09-15T03:40:48.256486+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":5518,"log_dups_size":0,"ondisk_log_size":5518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:17:29.191405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117440512,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":5499,"num_write_kb":248958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"24'4090","reported_seq":4195,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:48.341929+0000","last_change":"2024-09-15T03:36:48.162708+0000","last_active":"2024-09-15T03:40:48.341929+0000","last_peered":"2024-09-15T03:40:48.341929+0000","last_clean":"2024-09-15T03:40:48.341929+0000","last_became_active":"2024-09-15T03:36:48.162401+0000","last_became_peered":"2024-09-15T03:36:48.162401+0000","last_unstale":"2024-09-15T03:40:48.341929+0000","last_undegraded":"2024-09-15T03:40:48.341929+0000","last_fullsized":"2024-09-15T03:40:48.341929+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":4090,"log_dups_size":0,"ondisk_log_size":4090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:15:25.156037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104857600,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":310,"num_read_kb":242,"num_write":4350,"num_write_kb":177773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"24'3616","reported_seq":3646,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:48.333216+0000","last_change":"2024-09-15T03:36:48.162180+0000","last_active":"2024-09-15T03:40:48.333216+0000","last_peered":"2024-09-15T03:40:48.333216+0000","last_clean":"2024-09-15T03:40:48.333216+0000","last_became_active":"2024-09-15T03:36:48.161897+0000","last_became_peered":"2024-09-15T03:36:48.161897+0000","last_unstale":"2024-09-15T03:40:48.333216+0000","last_undegraded":"2024-09-15T03:40:48.333216+0000","last_fullsized":"2024-09-15T03:40:48.333216+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":3616,"log_dups_size":0,"ondisk_log_size":3616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:44:27.564730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":130023424,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":75,"num_read_kb":60,"num_write":3605,"num_write_kb":185220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"24'4055","reported_seq":4066,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:48.332171+0000","last_change":"2024-09-15T03:36:48.149972+0000","last_active":"2024-09-15T03:40:48.332171+0000","last_peered":"2024-09-15T03:40:48.332171+0000","last_clean":"2024-09-15T03:40:48.332171+0000","last_became_active":"2024-09-15T03:36:48.149651+0000","last_became_peered":"2024-09-15T03:36:48.149651+0000","last_unstale":"2024-09-15T03:40:48.332171+0000","last_undegraded":"2024-09-15T03:40:48.332171+0000","last_fullsized":"2024-09-15T03:40:48.332171+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":4055,"log_dups_size":0,"ondisk_log_size":4055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T15:32:38.282915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":111869952,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4006,"num_write_kb":198851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"22'2797","reported_seq":2824,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-15T03:39:32.410435+0000","last_change":"2024-09-15T03:36:48.159682+0000","last_active":"2024-09-15T03:39:32.410435+0000","last_peered":"2024-09-15T03:39:32.410435+0000","last_clean":"2024-09-15T03:39:32.410435+0000","last_became_active":"2024-09-15T03:36:48.159380+0000","last_became_peered":"2024-09-15T03:36:48.159380+0000","last_unstale":"2024-09-15T03:39:32.410435+0000","last_undegraded":"2024-09-15T03:39:32.410435+0000","last_fullsized":"2024-09-15T03:39:32.410435+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":2797,"log_dups_size":0,"ondisk_log_size":2797,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T13:43:53.575032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":50,"num_read_kb":41,"num_write":2819,"num_write_kb":144138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"24'5714","reported_seq":5730,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:48.324913+0000","last_change":"2024-09-15T03:36:48.151085+0000","last_active":"2024-09-15T03:40:48.324913+0000","last_peered":"2024-09-15T03:40:48.324913+0000","last_clean":"2024-09-15T03:40:48.324913+0000","last_became_active":"2024-09-15T03:36:48.150784+0000","last_became_peered":"2024-09-15T03:36:48.150784+0000","last_unstale":"2024-09-15T03:40:48.324913+0000","last_undegraded":"2024-09-15T03:40:48.324913+0000","last_fullsized":"2024-09-15T03:40:48.324913+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":5714,"log_dups_size":0,"ondisk_log_size":5714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:11:11.620798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":138543104,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":22,"num_read_kb":16,"num_write":5677,"num_write_kb":255624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"24'8354","reported_seq":8374,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:48.255488+0000","last_change":"2024-09-15T03:36:48.159758+0000","last_active":"2024-09-15T03:40:48.255488+0000","last_peered":"2024-09-15T03:40:48.255488+0000","last_clean":"2024-09-15T03:40:48.255488+0000","last_became_active":"2024-09-15T03:36:48.159470+0000","last_became_peered":"2024-09-15T03:36:48.159470+0000","last_unstale":"2024-09-15T03:40:48.255488+0000","last_undegraded":"2024-09-15T03:40:48.255488+0000","last_fullsized":"2024-09-15T03:40:48.255488+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":8354,"log_dups_size":0,"ondisk_log_size":8354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:06:20.271144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":126026248,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":36,"num_read_kb":28,"num_write":8349,"num_write_kb":200158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":3570163960,"num_objects":878,"num_object_clones":0,"num_object_copies":1756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":878,"num_whiteouts":0,"num_read":2563,"num_read_kb":2579,"num_write":159019,"num_write_kb":5633737,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":6842679296,"data_stored":6838412408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160792,"ondisk_log_size":160792,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542198,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1357616,"kb_used_data":1047784,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":93014224,"statfs":{"total":96636764160,"available":95246565376,"internally_reserved":0,"allocated":1072930816,"data_stored":1072149589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 03:40:48 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":1.5880000000000001,"15min":2.444},"min":{"1min":0.42099999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.262,"5min":35.189,"15min":35.189},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.77400000000000002,"15min":0.75},"min":{"1min":0.48999999999999999,"5min":0.245,"15min":0.245},"max":{"1min":1.2709999999999999,"5min":1.427,"15min":1.427},"last":0.95499999999999996}]},{"osd":1,"last update":"Sun Sep 15 03:40:48 2024","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":1.6299999999999999,"15min":2.4159999999999999},"min":{"1min":0.55700000000000005,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.1779999999999999,"5min":35.223999999999997,"15min":35.223999999999997},"last":0.97399999999999998},{"interface":"front","average":{"1min":0.88,"5min":0.81100000000000005,"15min":0.77500000000000002},"min":{"1min":0.70399999999999996,"5min":0.308,"15min":0.308},"max":{"1min":1.1619999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.59699999999999998}]},{"osd":2,"last update":"Sun Sep 15 03:40:48 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.76500000000000001,"15min":0.73999999999999999},"min":{"1min":0.63600000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.196,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.728,"15min":2.5150000000000001},"min":{"1min":0.79300000000000004,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.1990000000000001,"5min":35.283000000000001,"15min":35.283000000000001},"last":1.034}]},{"osd":3,"last update":"Sun Sep 15 03:40:48 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":0.84499999999999997,"15min":0.76100000000000001},"min":{"1min":0.64700000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.3360000000000001,"5min":1.71,"15min":1.71},"last":0.92500000000000004},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.8080000000000001,"15min":2.5699999999999998},"min":{"1min":0.88800000000000001,"5min":0.378,"15min":0.378},"max":{"1min":1.369,"5min":35.255000000000003,"15min":35.255000000000003},"last":1.101}]},{"osd":4,"last update":"Sun Sep 15 03:40:48 2024","interfaces":[{"interface":"back","average":{"1min":1.002,"5min":1.111,"15min":0.82899999999999996},"min":{"1min":0.69299999999999995,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.474,"5min":25.52,"15min":25.52},"last":0.76600000000000001},{"interface":"front","average":{"1min":1.024,"5min":0.871,"15min":0.79100000000000004},"min":{"1min":0.68899999999999995,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.2809999999999999,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.85199999999999998}]},{"osd":5,"last update":"Sun Sep 15 03:40:48 2024","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.8500000000000001,"15min":2.6480000000000001},"min":{"1min":0.78800000000000003,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.343,"5min":35.070999999999998,"15min":35.070999999999998},"last":1.083},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.101,"15min":1.107},"min":{"1min":1.075,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.421,"5min":5.2999999999999998,"15min":5.2999999999999998},"last":1.056}]},{"osd":6,"last update":"Sun Sep 15 03:40:48 2024","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.024,"15min":0.90400000000000003},"min":{"1min":1.085,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.429,"5min":2.5449999999999999,"15min":2.5449999999999999},"last":0.89800000000000002},{"interface":"front","average":{"1min":1.151,"5min":0.97299999999999998,"15min":0.86899999999999999},"min":{"1min":0.71499999999999997,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.452,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":1.004}]}]},{"osd":6,"up_from":14,"seq":60129542197,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":651352,"kb_used_data":349836,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93720488,"statfs":{"total":96636764160,"available":95969779712,"internally_reserved":0,"allocated":358232064,"data_stored":357835405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 03:40:43 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":1.0049999999999999,"15min":1.155},"min":{"1min":0.253,"5min":0.25,"15min":0.25},"max":{"1min":1.208,"5min":7.3170000000000002,"15min":7.3170000000000002},"last":1.1080000000000001},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.82399999999999995,"15min":0.77300000000000002},"min":{"1min":0.47799999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.212,"5min":7.3849999999999998,"15min":7.3849999999999998},"last":0.84099999999999997}]},{"osd":1,"last update":"Sun Sep 15 03:40:43 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.72899999999999998,"15min":0.69099999999999995},"min":{"1min":0.30299999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.157,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.89600000000000002,"15min":0.91800000000000004},"min":{"1min":0.45800000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.304,"5min":7.3559999999999999,"15min":7.3559999999999999},"last":0.88200000000000001}]},{"osd":2,"last update":"Sun Sep 15 03:40:43 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.92800000000000005,"15min":0.93500000000000005},"min":{"1min":0.436,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.286,"5min":7.4960000000000004,"15min":7.4960000000000004},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.84699999999999998,"15min":0.88600000000000001},"min":{"1min":0.49199999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.1499999999999999,"5min":1.575,"15min":1.575},"last":1.147}]},{"osd":3,"last update":"Sun Sep 15 03:40:43 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":1.0569999999999999,"15min":1.1499999999999999},"min":{"1min":0.42399999999999999,"5min":0.308,"15min":0.308},"max":{"1min":1.236,"5min":7.5869999999999997,"15min":7.5869999999999997},"last":1.03},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.2869999999999999,"15min":1.526},"min":{"1min":0.53800000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.423,"5min":11.494,"15min":11.494},"last":1.2}]},{"osd":4,"last update":"Sun Sep 15 03:40:43 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":1.0449999999999999,"15min":1.0820000000000001},"min":{"1min":0.32600000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.319,"5min":10.348000000000001,"15min":10.348000000000001},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.083,"5min":1.0509999999999999,"15min":1.0049999999999999},"min":{"1min":0.44,"5min":0.316,"15min":0.316},"max":{"1min":1.373,"5min":7.5620000000000003,"15min":7.5620000000000003},"last":1.385}]},{"osd":5,"last update":"Sun Sep 15 03:40:43 2024","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.86899999999999999,"15min":0.83799999999999997},"min":{"1min":0.54500000000000004,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.3400000000000001,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":1.3320000000000001},{"interface":"front","average":{"1min":1.147,"5min":1.0149999999999999,"15min":0.96699999999999997},"min":{"1min":0.56699999999999995,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.444,"5min":1.623,"15min":1.623},"last":1.5489999999999999}]},{"osd":7,"last update":"Sun Sep 15 03:40:43 2024","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.121,"15min":1.0149999999999999},"min":{"1min":0.40500000000000003,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.4279999999999999,"5min":10.923999999999999,"15min":10.923999999999999},"last":1.4399999999999999},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.1160000000000001,"15min":1.01},"min":{"1min":0.52100000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.7130000000000001,"5min":10.74,"15min":10.74},"last":1.522}]}]},{"osd":5,"up_from":14,"seq":60129542197,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1179936,"kb_used_data":874260,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":93191904,"statfs":{"total":96636764160,"available":95428509696,"internally_reserved":0,"allocated":895242240,"data_stored":894489264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 03:40:46 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.69699999999999995,"15min":0.66300000000000003},"min":{"1min":0.316,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.0049999999999999,"5min":1.3,"15min":1.3},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.80600000000000005,"15min":0.83899999999999997},"min":{"1min":0.36099999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.3300000000000001,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":0.73399999999999999}]},{"osd":1,"last update":"Sun Sep 15 03:40:46 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.78200000000000003,"15min":0.71599999999999997},"min":{"1min":0.33200000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.2450000000000001,"5min":2.302,"15min":2.302},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.73399999999999999,"15min":0.69599999999999995},"min":{"1min":0.36099999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.222,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.79200000000000004}]},{"osd":2,"last update":"Sun Sep 15 03:40:46 2024","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.85399999999999998,"15min":0.83499999999999996},"min":{"1min":0.48199999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":2.5779999999999998,"5min":2.5779999999999998,"15min":2.5779999999999998},"last":1.022},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.85499999999999998,"15min":0.85999999999999999},"min":{"1min":0.376,"5min":0.314,"15min":0.314},"max":{"1min":1.399,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":1.0569999999999999}]},{"osd":3,"last update":"Sun Sep 15 03:40:46 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":1.401,"15min":1.9419999999999999},"min":{"1min":0.438,"5min":0.375,"15min":0.375},"max":{"1min":1.2789999999999999,"5min":26.664999999999999,"15min":26.664999999999999},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.96999999999999997,"5min":1.538,"15min":2.1230000000000002},"min":{"1min":0.40500000000000003,"5min":0.312,"15min":0.312},"max":{"1min":1.327,"5min":26.748000000000001,"15min":26.748000000000001},"last":0.90600000000000003}]},{"osd":4,"last update":"Sun Sep 15 03:40:46 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.90600000000000003,"15min":0.88200000000000001},"min":{"1min":0.40600000000000003,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.3280000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.92800000000000005,"15min":0.88200000000000001},"min":{"1min":0.51100000000000001,"5min":0.318,"15min":0.318},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":1.1339999999999999}]},{"osd":6,"last update":"Sun Sep 15 03:40:46 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.97699999999999998,"15min":0.96099999999999997},"min":{"1min":0.52900000000000003,"5min":0.373,"15min":0.373},"max":{"1min":1.3320000000000001,"5min":1.9730000000000001,"15min":1.9730000000000001},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.056,"5min":1.04,"15min":1.004},"min":{"1min":0.436,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.452,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.1659999999999999}]},{"osd":7,"last update":"Sun Sep 15 03:40:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":0.97999999999999998,"15min":0.91400000000000003},"min":{"1min":0.47899999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.121,"15min":1.077},"min":{"1min":0.55500000000000005,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.6240000000000001,"5min":4.3070000000000004,"15min":4.3070000000000004},"last":1.258}]}]},{"osd":3,"up_from":14,"seq":60129542197,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1279284,"kb_used_data":981928,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93092556,"statfs":{"total":96636764160,"available":95326777344,"internally_reserved":0,"allocated":1005494272,"data_stored":1004659645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 03:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.63900000000000001,"15min":0.69699999999999995},"min":{"1min":0.28199999999999997,"5min":0.22,"15min":0.22},"max":{"1min":0.93200000000000005,"5min":4.7809999999999997,"15min":4.7809999999999997},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.79300000000000004,"15min":0.96899999999999997},"min":{"1min":0.29199999999999998,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":1.2609999999999999,"5min":7.7370000000000001,"15min":7.7370000000000001},"last":0.76200000000000001}]},{"osd":1,"last update":"Sun Sep 15 03:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.77900000000000003,"15min":0.63800000000000001},"min":{"1min":0.29099999999999998,"5min":0.217,"15min":0.217},"max":{"1min":1.032,"5min":13.749000000000001,"15min":13.749000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.83499999999999996,"5min":1.296,"15min":0.96799999999999997},"min":{"1min":0.44500000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.2989999999999999,"5min":46.225999999999999,"15min":46.225999999999999},"last":0.59199999999999997}]},{"osd":2,"last update":"Sun Sep 15 03:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.75700000000000001,"15min":0.71299999999999997},"min":{"1min":0.42399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.88,"5min":0.76200000000000001,"15min":0.72599999999999998},"min":{"1min":0.46899999999999997,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.167,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.93799999999999994}]},{"osd":4,"last update":"Sun Sep 15 03:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.93100000000000005,"15min":0.94299999999999995},"min":{"1min":0.56299999999999994,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.5660000000000001,"5min":2.9870000000000001,"15min":2.9870000000000001},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.016,"5min":0.95099999999999996,"15min":0.96799999999999997},"min":{"1min":0.38500000000000001,"5min":0.376,"15min":0.376},"max":{"1min":1.3380000000000001,"5min":2.609,"15min":2.609},"last":1.204}]},{"osd":5,"last update":"Sun Sep 15 03:40:45 2024","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":0.90900000000000003,"15min":0.85499999999999998},"min":{"1min":0.42499999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.6140000000000001,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":1.135},{"interface":"front","average":{"1min":1.155,"5min":1.071,"15min":1.1060000000000001},"min":{"1min":0.498,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.6679999999999999,"5min":5.0309999999999997,"15min":5.0309999999999997},"last":0.89600000000000002}]},{"osd":6,"last update":"Sun Sep 15 03:40:45 2024","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.014,"15min":0.98599999999999999},"min":{"1min":0.59899999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":1.024},{"interface":"front","average":{"1min":1.091,"5min":1.002,"15min":0.97199999999999998},"min":{"1min":0.61099999999999999,"5min":0.503,"15min":0.503},"max":{"1min":1.476,"5min":2.5550000000000002,"15min":2.5550000000000002},"last":1.2709999999999999}]},{"osd":7,"last update":"Sun Sep 15 03:40:45 2024","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.129,"15min":1.1830000000000001},"min":{"1min":0.74099999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.587,"5min":5.5439999999999996,"15min":5.5439999999999996},"last":1.3260000000000001},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.2989999999999999,"15min":1.5269999999999999},"min":{"1min":0.71899999999999997,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.5529999999999999,"5min":9.5850000000000009,"15min":9.5850000000000009},"last":1.296}]}]},{"osd":2,"up_from":14,"seq":60129542198,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":683092,"kb_used_data":377416,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":93688748,"statfs":{"total":96636764160,"available":95937277952,"internally_reserved":0,"allocated":386473984,"data_stored":386097365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 03:40:46 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.59699999999999998,"15min":0.61199999999999999},"min":{"1min":0.39800000000000002,"5min":0.193,"15min":0.193},"max":{"1min":0.96499999999999997,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.66700000000000004,"15min":0.65700000000000003},"min":{"1min":0.54000000000000004,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":1.228,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.63500000000000001}]},{"osd":1,"last update":"Sun Sep 15 03:40:46 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.73099999999999998,"15min":0.72599999999999998},"min":{"1min":0.47299999999999998,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.044,"5min":1.258,"15min":1.258},"last":0.876},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.61599999999999999,"15min":0.57999999999999996},"min":{"1min":0.41999999999999998,"5min":0.222,"15min":0.222},"max":{"1min":0.92800000000000005,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.66500000000000004}]},{"osd":3,"last update":"Sun Sep 15 03:40:46 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.68300000000000005,"15min":0.66000000000000003},"min":{"1min":0.42399999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.98299999999999998,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.77800000000000002,"15min":0.73399999999999999},"min":{"1min":0.40100000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.77300000000000002}]},{"osd":4,"last update":"Sun Sep 15 03:40:46 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.84099999999999997,"15min":0.77900000000000003},"min":{"1min":0.55500000000000005,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.02,"5min":0.91100000000000003,"15min":0.86599999999999999},"min":{"1min":0.53100000000000003,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.4239999999999999,"5min":1.573,"15min":1.573},"last":1.0149999999999999}]},{"osd":5,"last update":"Sun Sep 15 03:40:46 2024","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.94299999999999995,"15min":0.95399999999999996},"min":{"1min":0.68500000000000005,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.3680000000000001,"5min":4.5839999999999996,"15min":4.5839999999999996},"last":1.222},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0549999999999999,"15min":1.113},"min":{"1min":0.50900000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.341,"5min":4.8230000000000004,"15min":4.8230000000000004},"last":1.036}]},{"osd":6,"last update":"Sun Sep 15 03:40:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":0.94199999999999995,"15min":0.86799999999999999},"min":{"1min":0.81100000000000005,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.288,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":1.323},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.03,"15min":0.96999999999999997},"min":{"1min":0.49299999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.425,"5min":1.9039999999999999,"15min":1.9039999999999999},"last":1.0940000000000001}]},{"osd":7,"last update":"Sun Sep 15 03:40:46 2024","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.0389999999999999,"15min":1.044},"min":{"1min":0.72599999999999998,"5min":0.379,"15min":0.379},"max":{"1min":1.4590000000000001,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":1.2609999999999999},{"interface":"front","average":{"1min":1.169,"5min":1.3540000000000001,"15min":1.6080000000000001},"min":{"1min":0.74099999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.4770000000000001,"5min":10.428000000000001,"15min":10.428000000000001},"last":1.056}]}]},{"osd":4,"up_from":13,"seq":55834574902,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1618140,"kb_used_data":1320784,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":92753700,"statfs":{"total":96636764160,"available":94979788800,"internally_reserved":0,"allocated":1352482816,"data_stored":1351527913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 03:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.85799999999999998,"15min":0.93999999999999995},"min":{"1min":0.311,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.137,"5min":6.5010000000000003,"15min":6.5010000000000003},"last":1.089},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.85299999999999998,"15min":0.89300000000000002},"min":{"1min":0.34300000000000003,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.159,"5min":3.645,"15min":3.645},"last":0.73499999999999999}]},{"osd":1,"last update":"Sun Sep 15 03:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.84499999999999997,"15min":0.88900000000000001},"min":{"1min":0.29299999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.2589999999999999,"5min":3.7069999999999999,"15min":3.7069999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.81699999999999995,"15min":0.80200000000000005},"min":{"1min":0.44600000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.272,"5min":2.5779999999999998,"15min":2.5779999999999998},"last":1.3560000000000001}]},{"osd":2,"last update":"Sun Sep 15 03:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.84999999999999998,"15min":0.81100000000000005},"min":{"1min":0.45000000000000001,"5min":0.311,"15min":0.311},"max":{"1min":1.212,"5min":2.766,"15min":2.766},"last":1.121},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.84199999999999997,"15min":0.77200000000000002},"min":{"1min":0.375,"5min":0.316,"15min":0.316},"max":{"1min":1.256,"5min":3.2749999999999999,"15min":3.2749999999999999},"last":0.89700000000000002}]},{"osd":3,"last update":"Sun Sep 15 03:40:45 2024","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.0609999999999999,"15min":1.173},"min":{"1min":0.501,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.357,"5min":6.3620000000000001,"15min":6.3620000000000001},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.0640000000000001,"5min":0.94899999999999995,"15min":0.87},"min":{"1min":0.58799999999999997,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.4330000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":1.3089999999999999}]},{"osd":5,"last update":"Sun Sep 15 03:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":1.0069999999999999,"15min":1.0629999999999999},"min":{"1min":0.41899999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.6890000000000001,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.171,"5min":1.143,"15min":1.107},"min":{"1min":0.53000000000000003,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.651,"5min":4.1710000000000003,"15min":4.1710000000000003},"last":1.448}]},{"osd":6,"last update":"Sun Sep 15 03:40:45 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.0309999999999999,"15min":1.0389999999999999},"min":{"1min":0.45800000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.6120000000000001,"5min":3.669,"15min":3.669},"last":1.2370000000000001},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.0609999999999999,"15min":0.96399999999999997},"min":{"1min":0.73799999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.581,"5min":2.5329999999999999,"15min":2.5329999999999999},"last":1.403}]},{"osd":7,"last update":"Sun Sep 15 03:40:45 2024","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.018,"15min":0.92700000000000005},"min":{"1min":0.56599999999999995,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.498,"5min":3.238,"15min":3.238},"last":1.196},{"interface":"front","average":{"1min":1.244,"5min":1.1299999999999999,"15min":1.109},"min":{"1min":0.68700000000000006,"5min":0.378,"15min":0.378},"max":{"1min":1.552,"5min":3.8260000000000001,"15min":3.8260000000000001},"last":1.4930000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574901,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":986300,"kb_used_data":676464,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":93385540,"statfs":{"total":96636764160,"available":95626792960,"internally_reserved":0,"allocated":692699136,"data_stored":692171708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 03:40:47 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.58699999999999997,"15min":0.55100000000000005},"min":{"1min":0.27600000000000002,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.874,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.65900000000000003,"15min":0.60499999999999998},"min":{"1min":0.22900000000000001,"5min":0.187,"15min":0.187},"max":{"1min":0.91000000000000003,"5min":6.6050000000000004,"15min":6.6050000000000004},"last":0.65900000000000003}]},{"osd":2,"last update":"Sun Sep 15 03:40:47 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.64100000000000001,"15min":0.63100000000000001},"min":{"1min":0.314,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.88700000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.877},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.77900000000000003,"15min":0.751},"min":{"1min":0.32000000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.228,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":0.97099999999999997}]},{"osd":3,"last update":"Sun Sep 15 03:40:47 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.70599999999999996,"15min":0.64600000000000002},"min":{"1min":0.373,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":1.2350000000000001},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.85599999999999998,"15min":0.74199999999999999},"min":{"1min":0.40300000000000002,"5min":0.255,"15min":0.255},"max":{"1min":1.1659999999999999,"5min":5.6950000000000003,"15min":5.6950000000000003},"last":1.081}]},{"osd":4,"last update":"Sun Sep 15 03:40:47 2024","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.0409999999999999,"15min":0.90400000000000003},"min":{"1min":0.46000000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.3959999999999999,"5min":9.1080000000000005,"15min":9.1080000000000005},"last":1.3320000000000001},{"interface":"front","average":{"1min":1.149,"5min":1.1120000000000001,"15min":0.98799999999999999},"min":{"1min":0.59299999999999997,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.5660000000000001,"5min":9.1440000000000001,"15min":9.1440000000000001},"last":1.3779999999999999}]},{"osd":5,"last update":"Sun Sep 15 03:40:47 2024","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":0.93799999999999994,"15min":0.90100000000000002},"min":{"1min":0.47499999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.5349999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":1.113},{"interface":"front","average":{"1min":1.151,"5min":1.0289999999999999,"15min":0.97499999999999998},"min":{"1min":0.49099999999999999,"5min":0.314,"15min":0.314},"max":{"1min":1.4690000000000001,"5min":2.3580000000000001,"15min":2.3580000000000001},"last":1.425}]},{"osd":6,"last update":"Sun Sep 15 03:40:47 2024","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.0029999999999999,"15min":0.94099999999999995},"min":{"1min":0.436,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.387,"5min":6.4109999999999996,"15min":6.4109999999999996},"last":0.91300000000000003},{"interface":"front","average":{"1min":1.087,"5min":1.0489999999999999,"15min":1.006},"min":{"1min":0.53600000000000003,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.5229999999999999,"5min":5.3170000000000002,"15min":5.3170000000000002},"last":1.198}]},{"osd":7,"last update":"Sun Sep 15 03:40:47 2024","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.268,"15min":1.4279999999999999},"min":{"1min":0.55500000000000005,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.522,"5min":12.821,"15min":12.821},"last":1.143},{"interface":"front","average":{"1min":1.137,"5min":1.105,"15min":1.073},"min":{"1min":0.59099999999999997,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.498,"5min":5.2530000000000001,"15min":5.2530000000000001},"last":1.2829999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574902,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1362508,"kb_used_data":1056832,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":93009332,"statfs":{"total":96636764160,"available":95241555968,"internally_reserved":0,"allocated":1082195968,"data_stored":1081279613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Sep 15 03:40:41 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.59399999999999997,"15min":0.59999999999999998},"min":{"1min":0.20799999999999999,"5min":0.19,"15min":0.19},"max":{"1min":2.9279999999999999,"5min":2.9279999999999999,"15min":2.9279999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.58099999999999996,"15min":0.54400000000000004},"min":{"1min":0.41399999999999998,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.75900000000000001}]},{"osd":2,"last update":"Sun Sep 15 03:40:47 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.61899999999999999,"15min":0.55900000000000005},"min":{"1min":0.30399999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.69499999999999995,"15min":0.69899999999999995},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.1479999999999999,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":0.56200000000000006}]},{"osd":3,"last update":"Sun Sep 15 03:40:47 2024","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.73699999999999999,"15min":0.72999999999999998},"min":{"1min":0.47499999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":2.8860000000000001,"5min":2.8860000000000001,"15min":2.8860000000000001},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.69799999999999995,"15min":0.65800000000000003},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.71199999999999997}]},{"osd":4,"last update":"Sun Sep 15 03:40:41 2024","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.83099999999999996,"15min":0.80200000000000005},"min":{"1min":0.45900000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.1919999999999999,"5min":2.403,"15min":2.403},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.135,"5min":0.89700000000000002,"15min":0.83499999999999996},"min":{"1min":0.41299999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":3.1059999999999999,"5min":3.1059999999999999,"15min":3.1059999999999999},"last":0.73699999999999999}]},{"osd":5,"last update":"Sun Sep 15 03:40:47 2024","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":0.93000000000000005,"15min":0.84399999999999997},"min":{"1min":0.68899999999999995,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":2.9609999999999999,"5min":2.9609999999999999,"15min":2.9609999999999999},"last":0.94499999999999995},{"interface":"front","average":{"1min":1.1000000000000001,"5min":0.96599999999999997,"15min":0.88400000000000001},"min":{"1min":0.51200000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.54,"5min":2.4319999999999999,"15min":2.4319999999999999},"last":0.85399999999999998}]},{"osd":6,"last update":"Sun Sep 15 03:40:47 2024","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":0.92100000000000004,"15min":0.871},"min":{"1min":0.67500000000000004,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.3580000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.83099999999999996},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.0329999999999999,"15min":1.0089999999999999},"min":{"1min":0.56899999999999995,"5min":0.442,"15min":0.442},"max":{"1min":1.639,"5min":2.5459999999999998,"15min":2.5459999999999998},"last":0.77700000000000002}]},{"osd":7,"last update":"Sun Sep 15 03:40:47 2024","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.206,"15min":1.2989999999999999},"min":{"1min":0.54300000000000004,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.466,"5min":10.561999999999999,"15min":10.561999999999999},"last":0.97099999999999997},{"interface":"front","average":{"1min":1.254,"5min":0.98899999999999999,"15min":0.91100000000000003},"min":{"1min":0.64200000000000002,"5min":0.377,"15min":0.377},"max":{"1min":3.2120000000000002,"5min":3.2120000000000002,"15min":3.2120000000000002},"last":0.80900000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1081413632,"data_stored":1080658160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":692445184,"data_stored":692079164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":386224128,"data_stored":386004840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1005244416,"data_stored":1004567120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1352232960,"data_stored":1351435388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":894988288,"data_stored":894396720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":357982208,"data_stored":357742880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1072148480,"data_stored":1071528136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T03:40:55.473 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-15T03:36:47.712904+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=36, tm_sec=47, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=40, tm_sec=50, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T03:40:55.473 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-09-15T03:36:47.712904+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=36, tm_sec=47, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=40, tm_sec=50, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T03:40:55.473 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-15T03:36:47.712904+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=36, tm_sec=47, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=40, tm_sec=50, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T03:40:55.474 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-15T03:36:47.712904+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=36, tm_sec=47, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=40, tm_sec=50, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T03:40:55.474 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-09-15T03:36:47.712904+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=36, tm_sec=47, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=40, tm_sec=50, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T03:40:55.474 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-09-15T03:36:47.712904+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=36, tm_sec=47, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=40, tm_sec=50, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T03:40:55.474 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-09-15T03:36:47.712904+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=36, tm_sec=47, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=40, tm_sec=50, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T03:40:55.474 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-09-15T03:36:47.712904+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=36, tm_sec=47, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=40, tm_sec=50, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T03:40:55.474 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-09-15T03:36:47.712904+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=36, tm_sec=47, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=40, tm_sec=50, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T03:40:55.474 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-15T03:36:47.712904+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=36, tm_sec=47, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=40, tm_sec=50, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T03:40:55.474 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-15T03:36:47.712904+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=36, tm_sec=47, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=40, tm_sec=50, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T03:40:55.474 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-15T03:36:47.712904+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=36, tm_sec=47, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=40, tm_sec=50, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T03:40:55.475 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-15T03:36:39.639431+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=36, tm_sec=39, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=40, tm_sec=50, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T03:40:55.475 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-09-15T03:36:47.712904+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=36, tm_sec=47, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=40, tm_sec=50, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T03:40:55.475 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-15T03:36:39.639431+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=36, tm_sec=39, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=40, tm_sec=50, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T03:40:55.475 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-15T03:36:47.712904+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=36, tm_sec=47, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=40, tm_sec=50, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T03:40:55.475 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-15T03:36:39.639431+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=36, tm_sec=39, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=40, tm_sec=50, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T03:40:55.475 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-09-15T03:36:47.712904+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=36, tm_sec=47, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=40, tm_sec=50, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T03:40:55.475 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-15T03:36:39.639431+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=36, tm_sec=39, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=40, tm_sec=50, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T03:40:55.476 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-09-15T03:36:47.712904+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=36, tm_sec=47, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=40, tm_sec=50, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T03:40:55.476 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-15T03:36:47.712904+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=36, tm_sec=47, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=40, tm_sec=50, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T03:40:55.476 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-15T03:36:39.639431+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=36, tm_sec=39, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=40, tm_sec=50, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T03:40:55.476 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-15T03:36:47.712904+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=36, tm_sec=47, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=40, tm_sec=50, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T03:40:55.476 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-09-15T03:36:47.712904+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=36, tm_sec=47, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=40, tm_sec=50, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T03:40:55.476 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-15T03:36:39.639431+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=36, tm_sec=39, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=40, tm_sec=50, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T03:40:55.476 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-15T03:36:39.639431+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=36, tm_sec=39, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=40, tm_sec=50, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T03:40:55.476 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-15T03:36:37.618040+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=36, tm_sec=37, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=40, tm_sec=50, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T03:40:55.476 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-15T03:36:47.712904+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=36, tm_sec=47, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=40, tm_sec=50, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T03:40:55.477 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-15T03:36:47.712904+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=36, tm_sec=47, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=40, tm_sec=50, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T03:40:55.477 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-09-15T03:36:47.712904+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=36, tm_sec=47, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=40, tm_sec=50, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T03:40:55.477 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-15T03:36:47.712904+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=36, tm_sec=47, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=40, tm_sec=50, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T03:40:55.477 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-15T03:36:47.712904+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=36, tm_sec=47, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=40, tm_sec=50, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T03:40:55.477 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-15T03:36:47.712904+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=36, tm_sec=47, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=40, tm_sec=50, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T03:40:55.477 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-15T03:36:47.712904+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=36, tm_sec=47, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=40, tm_sec=50, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T03:40:55.478 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-15T03:36:47.712904+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=36, tm_sec=47, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=40, tm_sec=50, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T03:40:55.478 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-15T03:36:47.712904+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=36, tm_sec=47, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=40, tm_sec=50, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T03:40:55.478 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-15T03:36:47.712904+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=36, tm_sec=47, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=40, tm_sec=50, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T03:40:55.478 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-09-15T03:36:47.712904+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=36, tm_sec=47, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=40, tm_sec=50, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T03:40:55.478 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-15T03:36:47.712904+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=36, tm_sec=47, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=40, tm_sec=50, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T03:40:55.478 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-15T03:41:15.479 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T03:41:15.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.591+0000 7f485514b700 1 Processor -- start 2024-09-15T03:41:15.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.591+0000 7f485514b700 1 -- start start 2024-09-15T03:41:15.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.591+0000 7f485514b700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f485006f050 0x7f4850139780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:41:15.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.591+0000 7f485514b700 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48500aca30 con 0x7f485013af10 2024-09-15T03:41:15.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.591+0000 7f485514b700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f485006e850 con 0x7f48500ac660 2024-09-15T03:41:15.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.591+0000 7f485514b700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f485013b880 con 0x7f485006f050 2024-09-15T03:41:15.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.591+0000 7f485494a700 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7f485013af10 0x7f4850139cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:52238/0 (socket says 172.21.15.110:52238) 2024-09-15T03:41:15.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.591+0000 7f485494a700 1 -- 172.21.15.110:0/1555545226 learned_addr learned my addr 172.21.15.110:0/1555545226 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-15T03:41:15.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.591+0000 7f484f7fe700 1 --2- 172.21.15.110:0/1555545226 >> v2:172.21.15.177:3300/0 conn(0x7f485006f050 0x7f4850139780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:41:15.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.591+0000 7f484effd700 1 -- 172.21.15.110:0/1555545226 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3270794934 0 0) 0x7f48500aca30 con 0x7f485013af10 2024-09-15T03:41:15.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.591+0000 7f484effd700 1 -- 172.21.15.110:0/1555545226 --> v1:172.21.15.110:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f483c003570 con 0x7f485013af10 2024-09-15T03:41:15.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.591+0000 7f484f7fe700 1 -- 172.21.15.110:0/1555545226 >> v1:172.21.15.177:6789/0 conn(0x7f48500ac660 legacy=0x7f485013a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:41:15.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.591+0000 7f484f7fe700 1 -- 172.21.15.110:0/1555545226 >> v1:172.21.15.110:6789/0 conn(0x7f485013af10 legacy=0x7f4850139cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:41:15.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.591+0000 7f484f7fe700 1 -- 172.21.15.110:0/1555545226 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f485013b9c0 con 0x7f485006f050 2024-09-15T03:41:15.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.591+0000 7f484f7fe700 1 --2- 172.21.15.110:0/1555545226 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f485006f050 0x7f4850139780 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f48400098e0 tx=0x7f48400091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3580a8cd4d92ae15 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:41:15.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.591+0000 7f484effd700 1 -- 172.21.15.110:0/1555545226 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f484000b640 con 0x7f485006f050 2024-09-15T03:41:15.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.591+0000 7f484effd700 1 -- 172.21.15.110:0/1555545226 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f484000b7a0 con 0x7f485006f050 2024-09-15T03:41:15.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.591+0000 7f484effd700 1 -- 172.21.15.110:0/1555545226 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f484000bb40 con 0x7f485006f050 2024-09-15T03:41:15.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.595+0000 7f485514b700 1 -- 172.21.15.110:0/1555545226 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f485006f050 msgr2=0x7f4850139780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:41:15.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.595+0000 7f485514b700 1 --2- 172.21.15.110:0/1555545226 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f485006f050 0x7f4850139780 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f48400098e0 tx=0x7f48400091a0 comp rx=0 tx=0).stop 2024-09-15T03:41:15.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.595+0000 7f485514b700 1 -- 172.21.15.110:0/1555545226 shutdown_connections 2024-09-15T03:41:15.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.595+0000 7f485514b700 1 --2- 172.21.15.110:0/1555545226 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f485006f050 0x7f4850139780 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:41:15.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.595+0000 7f485514b700 1 -- 172.21.15.110:0/1555545226 >> 172.21.15.110:0/1555545226 conn(0x7f48500b0290 msgr2=0x7f48500b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:41:15.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.595+0000 7f485514b700 1 -- 172.21.15.110:0/1555545226 shutdown_connections 2024-09-15T03:41:15.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.595+0000 7f485514b700 1 -- 172.21.15.110:0/1555545226 wait complete. 2024-09-15T03:41:15.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.595+0000 7f485514b700 1 Processor -- start 2024-09-15T03:41:15.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.595+0000 7f485514b700 1 -- start start 2024-09-15T03:41:15.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.595+0000 7f485514b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f485006f050 0x7f485013ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:41:15.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.595+0000 7f485514b700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f48500ac660 0x7f485013bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:41:15.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.595+0000 7f485514b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f485013af10 0x7f485014d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:41:15.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.595+0000 7f485514b700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f485014d400 con 0x7f485013af10 2024-09-15T03:41:15.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.595+0000 7f485514b700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4850140ce0 con 0x7f485006f050 2024-09-15T03:41:15.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.595+0000 7f485514b700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f485013c500 con 0x7f48500ac660 2024-09-15T03:41:15.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.595+0000 7f485494a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f485013af10 0x7f485014d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:41:15.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.595+0000 7f484f7fe700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f48500ac660 0x7f485013bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:41:15.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.595+0000 7f485494a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f485013af10 0x7f485014d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55170/0 (socket says 172.21.15.110:55170) 2024-09-15T03:41:15.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.595+0000 7f485494a700 1 -- 172.21.15.110:0/3989609284 learned_addr learned my addr 172.21.15.110:0/3989609284 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:41:15.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.595+0000 7f484ffff700 1 --2- 172.21.15.110:0/3989609284 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f485006f050 0x7f485013ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:41:15.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.595+0000 7f485494a700 1 -- 172.21.15.110:0/3989609284 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f48500ac660 msgr2=0x7f485013bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:41:15.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.595+0000 7f485494a700 1 --2- 172.21.15.110:0/3989609284 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f48500ac660 0x7f485013bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:41:15.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.595+0000 7f485494a700 1 -- 172.21.15.110:0/3989609284 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f485006f050 msgr2=0x7f485013ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:41:15.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.595+0000 7f485494a700 1 --2- 172.21.15.110:0/3989609284 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f485006f050 0x7f485013ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:41:15.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.595+0000 7f485494a700 1 -- 172.21.15.110:0/3989609284 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4840009420 con 0x7f485013af10 2024-09-15T03:41:15.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.595+0000 7f485494a700 1 --2- 172.21.15.110:0/3989609284 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f485013af10 0x7f485014d890 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4848001460 tx=0x7f4848001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:41:15.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.595+0000 7f4847fff700 1 -- 172.21.15.110:0/3989609284 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f484800a870 con 0x7f485013af10 2024-09-15T03:41:15.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.595+0000 7f4847fff700 1 -- 172.21.15.110:0/3989609284 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f484800a9d0 con 0x7f485013af10 2024-09-15T03:41:15.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.595+0000 7f485514b700 1 -- 172.21.15.110:0/3989609284 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f485006ba00 con 0x7f485013af10 2024-09-15T03:41:15.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.595+0000 7f4847fff700 1 -- 172.21.15.110:0/3989609284 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f484800ad50 con 0x7f485013af10 2024-09-15T03:41:15.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.595+0000 7f485514b700 1 -- 172.21.15.110:0/3989609284 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f485014dff0 con 0x7f485013af10 2024-09-15T03:41:15.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.599+0000 7f485514b700 1 -- 172.21.15.110:0/3989609284 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f485014d1d0 con 0x7f485013af10 2024-09-15T03:41:15.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.599+0000 7f4847fff700 1 -- 172.21.15.110:0/3989609284 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) v1 ==== 85847+0+0 (secure 0 0 0) 0x7f4848007300 con 0x7f485013af10 2024-09-15T03:41:15.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.599+0000 7f4847fff700 1 --2- 172.21.15.110:0/3989609284 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f4830067a80 0x7f4830069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:41:15.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.599+0000 7f4847fff700 1 -- 172.21.15.110:0/3989609284 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5641+0+0 (secure 0 0 0) 0x7f4848085910 con 0x7f485013af10 2024-09-15T03:41:15.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.599+0000 7f484ffff700 1 --2- 172.21.15.110:0/3989609284 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f4830067a80 0x7f4830069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:41:15.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.603+0000 7f484ffff700 1 --2- 172.21.15.110:0/3989609284 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f4830067a80 0x7f4830069f30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4838003a80 tx=0x7f4838007ec0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:41:15.613 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.607+0000 7f4847fff700 1 -- 172.21.15.110:0/3989609284 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f485014d1d0 con 0x7f485013af10 2024-09-15T03:41:15.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.775+0000 7f485514b700 1 -- 172.21.15.110:0/3989609284 --> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f485014d1d0 con 0x7f4830067a80 2024-09-15T03:41:15.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.787+0000 7f4847fff700 1 -- 172.21.15.110:0/3989609284 <== mgr.4104 v2:172.21.15.110:6832/22190 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+154082 (secure 0 0 0) 0x7f485014d1d0 con 0x7f4830067a80 2024-09-15T03:41:15.794 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-15T03:41:15.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.791+0000 7f485514b700 1 -- 172.21.15.110:0/3989609284 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f4830067a80 msgr2=0x7f4830069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:41:15.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.791+0000 7f485514b700 1 --2- 172.21.15.110:0/3989609284 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f4830067a80 0x7f4830069f30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4838003a80 tx=0x7f4838007ec0 comp rx=0 tx=0).stop 2024-09-15T03:41:15.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.791+0000 7f485514b700 1 -- 172.21.15.110:0/3989609284 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f485013af10 msgr2=0x7f485014d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:41:15.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.791+0000 7f485514b700 1 --2- 172.21.15.110:0/3989609284 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f485013af10 0x7f485014d890 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4848001460 tx=0x7f4848001a40 comp rx=0 tx=0).stop 2024-09-15T03:41:15.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.791+0000 7f485514b700 1 -- 172.21.15.110:0/3989609284 shutdown_connections 2024-09-15T03:41:15.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.791+0000 7f485514b700 1 --2- 172.21.15.110:0/3989609284 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f4830067a80 0x7f4830069f30 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:41:15.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.791+0000 7f485514b700 1 --2- 172.21.15.110:0/3989609284 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f485013af10 0x7f485014d890 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:41:15.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.791+0000 7f485514b700 1 --2- 172.21.15.110:0/3989609284 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f48500ac660 0x7f485013bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:41:15.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.791+0000 7f485514b700 1 --2- 172.21.15.110:0/3989609284 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f485006f050 0x7f485013ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:41:15.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.791+0000 7f485514b700 1 -- 172.21.15.110:0/3989609284 >> 172.21.15.110:0/3989609284 conn(0x7f48500b0290 msgr2=0x7f48500ae950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:41:15.796 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.791+0000 7f485514b700 1 -- 172.21.15.110:0/3989609284 shutdown_connections 2024-09-15T03:41:15.797 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:15.791+0000 7f485514b700 1 -- 172.21.15.110:0/3989609284 wait complete. 2024-09-15T03:41:15.797 INFO:teuthology.orchestra.run.smithi110.stderr:dumped all 2024-09-15T03:41:15.809 INFO:teuthology.orchestra.run.smithi110.stdout:{"pg_ready":true,"pg_map":{"version":156,"stamp":"2024-09-15T03:41:14.684918+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3570754347,"num_objects":882,"num_object_clones":0,"num_object_copies":1764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":882,"num_whiteouts":0,"num_read":2629,"num_read_kb":2634,"num_write":159122,"num_write_kb":5635367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161577,"ondisk_log_size":161577,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":9126520,"kb_used_data":6685304,"kb_used_omap":13,"kb_used_meta":2441138,"kb_avail":745848200,"statfs":{"total":773094113280,"available":763748556800,"internally_reserved":0,"allocated":6845751296,"data_stored":6840210502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2499726016},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004041"},"pg_stats":[{"pgid":"3.1f","version":"24'3049","reported_seq":3071,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:41:03.738227+0000","last_change":"2024-09-15T03:41:03.738193+0000","last_active":"2024-09-15T03:41:03.738227+0000","last_peered":"2024-09-15T03:41:03.738227+0000","last_clean":"2024-09-15T03:41:03.738227+0000","last_became_active":"2024-09-15T03:36:48.151625+0000","last_became_peered":"2024-09-15T03:36:48.151625+0000","last_unstale":"2024-09-15T03:41:03.738227+0000","last_undegraded":"2024-09-15T03:41:03.738227+0000","last_fullsized":"2024-09-15T03:41:03.738227+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'3049","last_scrub_stamp":"2024-09-15T03:41:03.738112+0000","last_deep_scrub":"24'3049","last_deep_scrub_stamp":"2024-09-15T03:41:03.738112+0000","last_clean_scrub_stamp":"2024-09-15T03:41:03.738112+0000","objects_scrubbed":24,"log_size":3049,"log_dups_size":0,"ondisk_log_size":3049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:06:44.541343+0000","scrub_duration":0.28826884200000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663296,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3019,"num_write_kb":136067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"24'6236","reported_seq":6564,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:56.787862+0000","last_change":"2024-09-15T03:40:56.787830+0000","last_active":"2024-09-15T03:40:56.787862+0000","last_peered":"2024-09-15T03:40:56.787862+0000","last_clean":"2024-09-15T03:40:56.787862+0000","last_became_active":"2024-09-15T03:36:48.149731+0000","last_became_peered":"2024-09-15T03:36:48.149731+0000","last_unstale":"2024-09-15T03:40:56.787862+0000","last_undegraded":"2024-09-15T03:40:56.787862+0000","last_fullsized":"2024-09-15T03:40:56.787862+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'6236","last_scrub_stamp":"2024-09-15T03:40:56.787721+0000","last_deep_scrub":"24'6236","last_deep_scrub_stamp":"2024-09-15T03:40:56.787721+0000","last_clean_scrub_stamp":"2024-09-15T03:40:56.787721+0000","objects_scrubbed":28,"log_size":6236,"log_dups_size":0,"ondisk_log_size":6236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:19:35.805960+0000","scrub_duration":0.37018249800000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113377280,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":1022,"num_read_kb":916,"num_write":6394,"num_write_kb":178051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"24'6288","reported_seq":6403,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:48.336030+0000","last_change":"2024-09-15T03:36:48.162431+0000","last_active":"2024-09-15T03:40:48.336030+0000","last_peered":"2024-09-15T03:40:48.336030+0000","last_clean":"2024-09-15T03:40:48.336030+0000","last_became_active":"2024-09-15T03:36:48.162085+0000","last_became_peered":"2024-09-15T03:36:48.162085+0000","last_unstale":"2024-09-15T03:40:48.336030+0000","last_undegraded":"2024-09-15T03:40:48.336030+0000","last_fullsized":"2024-09-15T03:40:48.336030+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":6288,"log_dups_size":0,"ondisk_log_size":6288,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121898016,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":109,"num_read_kb":107,"num_write":6275,"num_write_kb":177699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"24'4678","reported_seq":4708,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:41:11.794374+0000","last_change":"2024-09-15T03:41:11.794328+0000","last_active":"2024-09-15T03:41:11.794374+0000","last_peered":"2024-09-15T03:41:11.794374+0000","last_clean":"2024-09-15T03:41:11.794374+0000","last_became_active":"2024-09-15T03:36:48.159272+0000","last_became_peered":"2024-09-15T03:36:48.159272+0000","last_unstale":"2024-09-15T03:41:11.794374+0000","last_undegraded":"2024-09-15T03:41:11.794374+0000","last_fullsized":"2024-09-15T03:41:11.794374+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'4678","last_scrub_stamp":"2024-09-15T03:41:11.794237+0000","last_deep_scrub":"24'4678","last_deep_scrub_stamp":"2024-09-15T03:41:11.794237+0000","last_clean_scrub_stamp":"2024-09-15T03:41:11.794237+0000","objects_scrubbed":22,"log_size":4678,"log_dups_size":0,"ondisk_log_size":4678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:52:12.546462+0000","scrub_duration":0.29342919200000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88866816,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":4665,"num_write_kb":169740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"24'4826","reported_seq":4849,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:56.758451+0000","last_change":"2024-09-15T03:40:56.758405+0000","last_active":"2024-09-15T03:40:56.758451+0000","last_peered":"2024-09-15T03:40:56.758451+0000","last_clean":"2024-09-15T03:40:56.758451+0000","last_became_active":"2024-09-15T03:36:48.150057+0000","last_became_peered":"2024-09-15T03:36:48.150057+0000","last_unstale":"2024-09-15T03:40:56.758451+0000","last_undegraded":"2024-09-15T03:40:56.758451+0000","last_fullsized":"2024-09-15T03:40:56.758451+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'4826","last_scrub_stamp":"2024-09-15T03:40:56.758284+0000","last_deep_scrub":"24'4826","last_deep_scrub_stamp":"2024-09-15T03:40:56.758284+0000","last_clean_scrub_stamp":"2024-09-15T03:40:56.758284+0000","objects_scrubbed":28,"log_size":4826,"log_dups_size":0,"ondisk_log_size":4826,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T12:47:57.646441+0000","scrub_duration":0.33452131800000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117440512,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4750,"num_write_kb":185221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"24'4732","reported_seq":4759,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:55.144629+0000","last_change":"2024-09-15T03:40:55.144557+0000","last_active":"2024-09-15T03:40:55.144629+0000","last_peered":"2024-09-15T03:40:55.144629+0000","last_clean":"2024-09-15T03:40:55.144629+0000","last_became_active":"2024-09-15T03:36:48.144097+0000","last_became_peered":"2024-09-15T03:36:48.144097+0000","last_unstale":"2024-09-15T03:40:55.144629+0000","last_undegraded":"2024-09-15T03:40:55.144629+0000","last_fullsized":"2024-09-15T03:40:55.144629+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'4732","last_scrub_stamp":"2024-09-15T03:40:55.144302+0000","last_deep_scrub":"24'4732","last_deep_scrub_stamp":"2024-09-15T03:40:55.144302+0000","last_clean_scrub_stamp":"2024-09-15T03:40:55.144302+0000","objects_scrubbed":18,"log_size":4732,"log_dups_size":0,"ondisk_log_size":4732,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:12:19.069563+0000","scrub_duration":0.215515611,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68681728,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":22,"num_read_kb":16,"num_write":4699,"num_write_kb":143618,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"24'4521","reported_seq":4545,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:41:07.568108+0000","last_change":"2024-09-15T03:41:07.568045+0000","last_active":"2024-09-15T03:41:07.568108+0000","last_peered":"2024-09-15T03:41:07.568108+0000","last_clean":"2024-09-15T03:41:07.568108+0000","last_became_active":"2024-09-15T03:36:48.142085+0000","last_became_peered":"2024-09-15T03:36:48.142085+0000","last_unstale":"2024-09-15T03:41:07.568108+0000","last_undegraded":"2024-09-15T03:41:07.568108+0000","last_fullsized":"2024-09-15T03:41:07.568108+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'4521","last_scrub_stamp":"2024-09-15T03:41:07.567893+0000","last_deep_scrub":"24'4521","last_deep_scrub_stamp":"2024-09-15T03:41:07.567893+0000","last_clean_scrub_stamp":"2024-09-15T03:41:07.567893+0000","objects_scrubbed":28,"log_size":4521,"log_dups_size":0,"ondisk_log_size":4521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:13:47.082289+0000","scrub_duration":0.23014422900000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":111872344,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":4448,"num_write_kb":183583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"24'7233","reported_seq":7260,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:41:03.795098+0000","last_change":"2024-09-15T03:41:03.795039+0000","last_active":"2024-09-15T03:41:03.795098+0000","last_peered":"2024-09-15T03:41:03.795098+0000","last_clean":"2024-09-15T03:41:03.795098+0000","last_became_active":"2024-09-15T03:36:48.138049+0000","last_became_peered":"2024-09-15T03:36:48.138049+0000","last_unstale":"2024-09-15T03:41:03.795098+0000","last_undegraded":"2024-09-15T03:41:03.795098+0000","last_fullsized":"2024-09-15T03:41:03.795098+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'7233","last_scrub_stamp":"2024-09-15T03:41:03.794874+0000","last_deep_scrub":"24'7233","last_deep_scrub_stamp":"2024-09-15T03:41:03.794874+0000","last_clean_scrub_stamp":"2024-09-15T03:41:03.794874+0000","objects_scrubbed":28,"log_size":7233,"log_dups_size":0,"ondisk_log_size":7233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T13:50:20.679143+0000","scrub_duration":0.27943716800000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113246720,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":15,"num_read_kb":11,"num_write":7218,"num_write_kb":193736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"24'7897","reported_seq":7927,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:55.869062+0000","last_change":"2024-09-15T03:40:55.869035+0000","last_active":"2024-09-15T03:40:55.869062+0000","last_peered":"2024-09-15T03:40:55.869062+0000","last_clean":"2024-09-15T03:40:55.869062+0000","last_became_active":"2024-09-15T03:36:48.149792+0000","last_became_peered":"2024-09-15T03:36:48.149792+0000","last_unstale":"2024-09-15T03:40:55.869062+0000","last_undegraded":"2024-09-15T03:40:55.869062+0000","last_fullsized":"2024-09-15T03:40:55.869062+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'7897","last_scrub_stamp":"2024-09-15T03:40:55.868958+0000","last_deep_scrub":"24'7897","last_deep_scrub_stamp":"2024-09-15T03:40:55.868958+0000","last_clean_scrub_stamp":"2024-09-15T03:40:55.868958+0000","objects_scrubbed":35,"log_size":7897,"log_dups_size":0,"ondisk_log_size":7897,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T13:15:33.314556+0000","scrub_duration":0.44187000399999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":146800640,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":7874,"num_write_kb":194640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"24'2428","reported_seq":2453,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:41:03.743187+0000","last_change":"2024-09-15T03:41:03.743157+0000","last_active":"2024-09-15T03:41:03.743187+0000","last_peered":"2024-09-15T03:41:03.743187+0000","last_clean":"2024-09-15T03:41:03.743187+0000","last_became_active":"2024-09-15T03:36:48.154800+0000","last_became_peered":"2024-09-15T03:36:48.154800+0000","last_unstale":"2024-09-15T03:41:03.743187+0000","last_undegraded":"2024-09-15T03:41:03.743187+0000","last_fullsized":"2024-09-15T03:41:03.743187+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'2428","last_scrub_stamp":"2024-09-15T03:41:03.743030+0000","last_deep_scrub":"24'2428","last_deep_scrub_stamp":"2024-09-15T03:41:03.743030+0000","last_clean_scrub_stamp":"2024-09-15T03:41:03.743030+0000","objects_scrubbed":19,"log_size":2428,"log_dups_size":0,"ondisk_log_size":2428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:57:26.522200+0000","scrub_duration":0.22485928499999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":2388,"num_write_kb":127362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"23'6301","reported_seq":6319,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:48.255666+0000","last_change":"2024-09-15T03:36:48.152499+0000","last_active":"2024-09-15T03:40:48.255666+0000","last_peered":"2024-09-15T03:40:48.255666+0000","last_clean":"2024-09-15T03:40:48.255666+0000","last_became_active":"2024-09-15T03:36:48.152191+0000","last_became_peered":"2024-09-15T03:36:48.152191+0000","last_unstale":"2024-09-15T03:40:48.255666+0000","last_undegraded":"2024-09-15T03:40:48.255666+0000","last_fullsized":"2024-09-15T03:40:48.255666+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":6301,"log_dups_size":0,"ondisk_log_size":6301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":124518928,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":19,"num_read_kb":14,"num_write":6316,"num_write_kb":168780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"24'5415","reported_seq":5441,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:56.272066+0000","last_change":"2024-09-15T03:40:56.272033+0000","last_active":"2024-09-15T03:40:56.272066+0000","last_peered":"2024-09-15T03:40:56.272066+0000","last_clean":"2024-09-15T03:40:56.272066+0000","last_became_active":"2024-09-15T03:36:48.153926+0000","last_became_peered":"2024-09-15T03:36:48.153926+0000","last_unstale":"2024-09-15T03:40:56.272066+0000","last_undegraded":"2024-09-15T03:40:56.272066+0000","last_fullsized":"2024-09-15T03:40:56.272066+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'5415","last_scrub_stamp":"2024-09-15T03:40:56.271945+0000","last_deep_scrub":"24'5415","last_deep_scrub_stamp":"2024-09-15T03:40:56.271945+0000","last_clean_scrub_stamp":"2024-09-15T03:40:56.271945+0000","objects_scrubbed":28,"log_size":5415,"log_dups_size":0,"ondisk_log_size":5415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:00:32.277556+0000","scrub_duration":0.37861801299999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109121016,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":5342,"num_write_kb":144586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":31,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:58.377582+0000","last_change":"2024-09-15T03:40:58.377503+0000","last_active":"2024-09-15T03:40:58.377582+0000","last_peered":"2024-09-15T03:40:58.377582+0000","last_clean":"2024-09-15T03:40:58.377582+0000","last_became_active":"2024-09-15T03:36:40.672828+0000","last_became_peered":"2024-09-15T03:36:40.672828+0000","last_unstale":"2024-09-15T03:40:58.377582+0000","last_undegraded":"2024-09-15T03:40:58.377582+0000","last_fullsized":"2024-09-15T03:40:58.377582+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:40:58.377270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:40:58.377270+0000","last_clean_scrub_stamp":"2024-09-15T03:40:58.377270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:22:36.982400+0000","scrub_duration":0.003092628,"objects_trimmed":0,"snaptrim_duration":0.00064069099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"24'6015","reported_seq":6061,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:53.674105+0000","last_change":"2024-09-15T03:40:53.674065+0000","last_active":"2024-09-15T03:40:53.674105+0000","last_peered":"2024-09-15T03:40:53.674105+0000","last_clean":"2024-09-15T03:40:53.674105+0000","last_became_active":"2024-09-15T03:36:48.143177+0000","last_became_peered":"2024-09-15T03:36:48.143177+0000","last_unstale":"2024-09-15T03:40:53.674105+0000","last_undegraded":"2024-09-15T03:40:53.674105+0000","last_fullsized":"2024-09-15T03:40:53.674105+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'6015","last_scrub_stamp":"2024-09-15T03:40:53.673943+0000","last_deep_scrub":"24'6015","last_deep_scrub_stamp":"2024-09-15T03:40:53.673943+0000","last_clean_scrub_stamp":"2024-09-15T03:40:53.673943+0000","objects_scrubbed":33,"log_size":6015,"log_dups_size":0,"ondisk_log_size":6015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T13:33:53.399179+0000","scrub_duration":0.274737495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":134639616,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":92,"num_read_kb":80,"num_write":5989,"num_write_kb":192351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":31,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:53.212546+0000","last_change":"2024-09-15T03:40:53.212457+0000","last_active":"2024-09-15T03:40:53.212546+0000","last_peered":"2024-09-15T03:40:53.212546+0000","last_clean":"2024-09-15T03:40:53.212546+0000","last_became_active":"2024-09-15T03:36:40.671677+0000","last_became_peered":"2024-09-15T03:36:40.671677+0000","last_unstale":"2024-09-15T03:40:53.212546+0000","last_undegraded":"2024-09-15T03:40:53.212546+0000","last_fullsized":"2024-09-15T03:40:53.212546+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:40:53.212248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:40:53.212248+0000","last_clean_scrub_stamp":"2024-09-15T03:40:53.212248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:33:29.916166+0000","scrub_duration":0.003136242,"objects_trimmed":0,"snaptrim_duration":0.000861089,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"24'5430","reported_seq":5453,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:58.788643+0000","last_change":"2024-09-15T03:40:58.788611+0000","last_active":"2024-09-15T03:40:58.788643+0000","last_peered":"2024-09-15T03:40:58.788643+0000","last_clean":"2024-09-15T03:40:58.788643+0000","last_became_active":"2024-09-15T03:36:48.151844+0000","last_became_peered":"2024-09-15T03:36:48.151844+0000","last_unstale":"2024-09-15T03:40:58.788643+0000","last_undegraded":"2024-09-15T03:40:58.788643+0000","last_fullsized":"2024-09-15T03:40:58.788643+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'5430","last_scrub_stamp":"2024-09-15T03:40:58.788521+0000","last_deep_scrub":"24'5430","last_deep_scrub_stamp":"2024-09-15T03:40:58.788521+0000","last_clean_scrub_stamp":"2024-09-15T03:40:58.788521+0000","objects_scrubbed":28,"log_size":5430,"log_dups_size":0,"ondisk_log_size":5430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:42:39.962811+0000","scrub_duration":0.34940842100000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":115998720,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5352,"num_write_kb":174984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":31,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:41:09.338334+0000","last_change":"2024-09-15T03:41:09.338237+0000","last_active":"2024-09-15T03:41:09.338334+0000","last_peered":"2024-09-15T03:41:09.338334+0000","last_clean":"2024-09-15T03:41:09.338334+0000","last_became_active":"2024-09-15T03:36:41.079216+0000","last_became_peered":"2024-09-15T03:36:41.079216+0000","last_unstale":"2024-09-15T03:41:09.338334+0000","last_undegraded":"2024-09-15T03:41:09.338334+0000","last_fullsized":"2024-09-15T03:41:09.338334+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:41:09.338037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:41:09.338037+0000","last_clean_scrub_stamp":"2024-09-15T03:41:09.338037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T13:23:59.168130+0000","scrub_duration":0.0036244739999999999,"objects_trimmed":0,"snaptrim_duration":0.00062568200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"24'7307","reported_seq":7341,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:41:06.686821+0000","last_change":"2024-09-15T03:41:06.686777+0000","last_active":"2024-09-15T03:41:06.686821+0000","last_peered":"2024-09-15T03:41:06.686821+0000","last_clean":"2024-09-15T03:41:06.686821+0000","last_became_active":"2024-09-15T03:36:48.216337+0000","last_became_peered":"2024-09-15T03:36:48.216337+0000","last_unstale":"2024-09-15T03:41:06.686821+0000","last_undegraded":"2024-09-15T03:41:06.686821+0000","last_fullsized":"2024-09-15T03:41:06.686821+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'7307","last_scrub_stamp":"2024-09-15T03:41:06.686644+0000","last_deep_scrub":"24'7307","last_deep_scrub_stamp":"2024-09-15T03:41:06.686644+0000","last_clean_scrub_stamp":"2024-09-15T03:41:06.686644+0000","objects_scrubbed":37,"log_size":7307,"log_dups_size":0,"ondisk_log_size":7307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:19:17.191909+0000","scrub_duration":0.35128014299999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":146931712,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":45,"num_read_kb":34,"num_write":7284,"num_write_kb":246402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":31,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:58.302534+0000","last_change":"2024-09-15T03:40:58.302464+0000","last_active":"2024-09-15T03:40:58.302534+0000","last_peered":"2024-09-15T03:40:58.302534+0000","last_clean":"2024-09-15T03:40:58.302534+0000","last_became_active":"2024-09-15T03:36:41.077266+0000","last_became_peered":"2024-09-15T03:36:41.077266+0000","last_unstale":"2024-09-15T03:40:58.302534+0000","last_undegraded":"2024-09-15T03:40:58.302534+0000","last_fullsized":"2024-09-15T03:40:58.302534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:40:58.302274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:40:58.302274+0000","last_clean_scrub_stamp":"2024-09-15T03:40:58.302274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:00:53.493431+0000","scrub_duration":0.0031430619999999999,"objects_trimmed":0,"snaptrim_duration":0.00067514400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"24'3304","reported_seq":3326,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:41:00.727165+0000","last_change":"2024-09-15T03:41:00.727138+0000","last_active":"2024-09-15T03:41:00.727165+0000","last_peered":"2024-09-15T03:41:00.727165+0000","last_clean":"2024-09-15T03:41:00.727165+0000","last_became_active":"2024-09-15T03:36:48.151520+0000","last_became_peered":"2024-09-15T03:36:48.151520+0000","last_unstale":"2024-09-15T03:41:00.727165+0000","last_undegraded":"2024-09-15T03:41:00.727165+0000","last_fullsized":"2024-09-15T03:41:00.727165+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'3304","last_scrub_stamp":"2024-09-15T03:41:00.727061+0000","last_deep_scrub":"24'3304","last_deep_scrub_stamp":"2024-09-15T03:41:00.727061+0000","last_clean_scrub_stamp":"2024-09-15T03:41:00.727061+0000","objects_scrubbed":22,"log_size":3304,"log_dups_size":0,"ondisk_log_size":3304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T15:01:42.696558+0000","scrub_duration":0.25260938199999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3278,"num_write_kb":144390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"24'3476","reported_seq":3513,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:58.195729+0000","last_change":"2024-09-15T03:40:58.195683+0000","last_active":"2024-09-15T03:40:58.195729+0000","last_peered":"2024-09-15T03:40:58.195729+0000","last_clean":"2024-09-15T03:40:58.195729+0000","last_became_active":"2024-09-15T03:36:48.145845+0000","last_became_peered":"2024-09-15T03:36:48.145845+0000","last_unstale":"2024-09-15T03:40:58.195729+0000","last_undegraded":"2024-09-15T03:40:58.195729+0000","last_fullsized":"2024-09-15T03:40:58.195729+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'3476","last_scrub_stamp":"2024-09-15T03:40:58.195566+0000","last_deep_scrub":"24'3476","last_deep_scrub_stamp":"2024-09-15T03:40:58.195566+0000","last_clean_scrub_stamp":"2024-09-15T03:40:58.195566+0000","objects_scrubbed":27,"log_size":3476,"log_dups_size":0,"ondisk_log_size":3476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:57:26.163454+0000","scrub_duration":0.311787747,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113246208,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":50,"num_read_kb":41,"num_write":3479,"num_write_kb":144783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"24'3","reported_seq":34,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:57.489841+0000","last_change":"2024-09-15T03:40:57.489767+0000","last_active":"2024-09-15T03:40:57.489841+0000","last_peered":"2024-09-15T03:40:57.489841+0000","last_clean":"2024-09-15T03:40:57.489841+0000","last_became_active":"2024-09-15T03:36:40.673743+0000","last_became_peered":"2024-09-15T03:36:40.673743+0000","last_unstale":"2024-09-15T03:40:57.489841+0000","last_undegraded":"2024-09-15T03:40:57.489841+0000","last_fullsized":"2024-09-15T03:40:57.489841+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'3","last_scrub_stamp":"2024-09-15T03:40:57.489545+0000","last_deep_scrub":"24'3","last_deep_scrub_stamp":"2024-09-15T03:40:57.489545+0000","last_clean_scrub_stamp":"2024-09-15T03:40:57.489545+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:18:35.777966+0000","scrub_duration":0.006897373,"objects_trimmed":0,"snaptrim_duration":0.002159518,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"24'4749","reported_seq":4786,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:41:00.645265+0000","last_change":"2024-09-15T03:41:00.645216+0000","last_active":"2024-09-15T03:41:00.645265+0000","last_peered":"2024-09-15T03:41:00.645265+0000","last_clean":"2024-09-15T03:41:00.645265+0000","last_became_active":"2024-09-15T03:36:48.216269+0000","last_became_peered":"2024-09-15T03:36:48.216269+0000","last_unstale":"2024-09-15T03:41:00.645265+0000","last_undegraded":"2024-09-15T03:41:00.645265+0000","last_fullsized":"2024-09-15T03:41:00.645265+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'4749","last_scrub_stamp":"2024-09-15T03:41:00.645075+0000","last_deep_scrub":"24'4749","last_deep_scrub_stamp":"2024-09-15T03:41:00.645075+0000","last_clean_scrub_stamp":"2024-09-15T03:41:00.645075+0000","objects_scrubbed":37,"log_size":4749,"log_dups_size":0,"ondisk_log_size":4749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T15:04:15.528288+0000","scrub_duration":0.30872137900000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":142877200,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":51,"num_read_kb":164,"num_write":4698,"num_write_kb":182422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":31,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:53.651919+0000","last_change":"2024-09-15T03:40:53.651875+0000","last_active":"2024-09-15T03:40:53.651919+0000","last_peered":"2024-09-15T03:40:53.651919+0000","last_clean":"2024-09-15T03:40:53.651919+0000","last_became_active":"2024-09-15T03:36:40.669392+0000","last_became_peered":"2024-09-15T03:36:40.669392+0000","last_unstale":"2024-09-15T03:40:53.651919+0000","last_undegraded":"2024-09-15T03:40:53.651919+0000","last_fullsized":"2024-09-15T03:40:53.651919+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:40:53.651715+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:40:53.651715+0000","last_clean_scrub_stamp":"2024-09-15T03:40:53.651715+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:08:35.720046+0000","scrub_duration":0.0021291890000000001,"objects_trimmed":0,"snaptrim_duration":0.00059629799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"24'3251","reported_seq":3281,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:54.648696+0000","last_change":"2024-09-15T03:40:54.648656+0000","last_active":"2024-09-15T03:40:54.648696+0000","last_peered":"2024-09-15T03:40:54.648696+0000","last_clean":"2024-09-15T03:40:54.648696+0000","last_became_active":"2024-09-15T03:36:48.144313+0000","last_became_peered":"2024-09-15T03:36:48.144313+0000","last_unstale":"2024-09-15T03:40:54.648696+0000","last_undegraded":"2024-09-15T03:40:54.648696+0000","last_fullsized":"2024-09-15T03:40:54.648696+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'3251","last_scrub_stamp":"2024-09-15T03:40:54.648582+0000","last_deep_scrub":"24'3251","last_deep_scrub_stamp":"2024-09-15T03:40:54.648582+0000","last_clean_scrub_stamp":"2024-09-15T03:40:54.648582+0000","objects_scrubbed":23,"log_size":3251,"log_dups_size":0,"ondisk_log_size":3251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:24:02.179680+0000","scrub_duration":0.24652201300000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92405760,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":33,"num_read_kb":87,"num_write":3327,"num_write_kb":128260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":31,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:41:07.352881+0000","last_change":"2024-09-15T03:41:07.352809+0000","last_active":"2024-09-15T03:41:07.352881+0000","last_peered":"2024-09-15T03:41:07.352881+0000","last_clean":"2024-09-15T03:41:07.352881+0000","last_became_active":"2024-09-15T03:36:40.672416+0000","last_became_peered":"2024-09-15T03:36:40.672416+0000","last_unstale":"2024-09-15T03:41:07.352881+0000","last_undegraded":"2024-09-15T03:41:07.352881+0000","last_fullsized":"2024-09-15T03:41:07.352881+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:41:07.352573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:41:07.352573+0000","last_clean_scrub_stamp":"2024-09-15T03:41:07.352573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:17:17.658369+0000","scrub_duration":0.0038595510000000001,"objects_trimmed":0,"snaptrim_duration":0.00065187600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"24'3123","reported_seq":3145,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:53.585199+0000","last_change":"2024-09-15T03:40:53.585158+0000","last_active":"2024-09-15T03:40:53.585199+0000","last_peered":"2024-09-15T03:40:53.585199+0000","last_clean":"2024-09-15T03:40:53.585199+0000","last_became_active":"2024-09-15T03:36:48.150693+0000","last_became_peered":"2024-09-15T03:36:48.150693+0000","last_unstale":"2024-09-15T03:40:53.585199+0000","last_undegraded":"2024-09-15T03:40:53.585199+0000","last_fullsized":"2024-09-15T03:40:53.585199+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'3123","last_scrub_stamp":"2024-09-15T03:40:53.585070+0000","last_deep_scrub":"24'3123","last_deep_scrub_stamp":"2024-09-15T03:40:53.585070+0000","last_clean_scrub_stamp":"2024-09-15T03:40:53.585070+0000","objects_scrubbed":16,"log_size":3123,"log_dups_size":0,"ondisk_log_size":3123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:13:52.170594+0000","scrub_duration":0.174875789,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67108864,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3105,"num_write_kb":103402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"24'2","reported_seq":33,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:58.467469+0000","last_change":"2024-09-15T03:40:58.467398+0000","last_active":"2024-09-15T03:40:58.467469+0000","last_peered":"2024-09-15T03:40:58.467469+0000","last_clean":"2024-09-15T03:40:58.467469+0000","last_became_active":"2024-09-15T03:36:40.674442+0000","last_became_peered":"2024-09-15T03:36:40.674442+0000","last_unstale":"2024-09-15T03:40:58.467469+0000","last_undegraded":"2024-09-15T03:40:58.467469+0000","last_fullsized":"2024-09-15T03:40:58.467469+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'2","last_scrub_stamp":"2024-09-15T03:40:58.467214+0000","last_deep_scrub":"24'2","last_deep_scrub_stamp":"2024-09-15T03:40:58.467214+0000","last_clean_scrub_stamp":"2024-09-15T03:40:58.467214+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:42:10.462331+0000","scrub_duration":0.004548524,"objects_trimmed":0,"snaptrim_duration":0.00067001500000000004,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"24'74","reported_seq":120,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:59.495959+0000","last_change":"2024-09-15T03:40:59.495892+0000","last_active":"2024-09-15T03:40:59.495959+0000","last_peered":"2024-09-15T03:40:59.495959+0000","last_clean":"2024-09-15T03:40:59.495959+0000","last_became_active":"2024-09-15T03:36:38.813490+0000","last_became_peered":"2024-09-15T03:36:38.813490+0000","last_unstale":"2024-09-15T03:40:59.495959+0000","last_undegraded":"2024-09-15T03:40:59.495959+0000","last_fullsized":"2024-09-15T03:40:59.495959+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'74","last_scrub_stamp":"2024-09-15T03:40:59.495711+0000","last_deep_scrub":"24'74","last_deep_scrub_stamp":"2024-09-15T03:40:59.495711+0000","last_clean_scrub_stamp":"2024-09-15T03:40:59.495711+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:54:47.135962+0000","scrub_duration":0.0079286540000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"24'5218","reported_seq":5245,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:59.654581+0000","last_change":"2024-09-15T03:40:59.654544+0000","last_active":"2024-09-15T03:40:59.654581+0000","last_peered":"2024-09-15T03:40:59.654581+0000","last_clean":"2024-09-15T03:40:59.654581+0000","last_became_active":"2024-09-15T03:36:48.142313+0000","last_became_peered":"2024-09-15T03:36:48.142313+0000","last_unstale":"2024-09-15T03:40:59.654581+0000","last_undegraded":"2024-09-15T03:40:59.654581+0000","last_fullsized":"2024-09-15T03:40:59.654581+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'5218","last_scrub_stamp":"2024-09-15T03:40:59.654440+0000","last_deep_scrub":"24'5218","last_deep_scrub_stamp":"2024-09-15T03:40:59.654440+0000","last_clean_scrub_stamp":"2024-09-15T03:40:59.654440+0000","objects_scrubbed":28,"log_size":5218,"log_dups_size":0,"ondisk_log_size":5218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T12:51:50.746216+0000","scrub_duration":0.26478074600000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":116760576,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":12,"num_read_kb":135,"num_write":5130,"num_write_kb":177200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"24'7969","reported_seq":8107,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:54.221970+0000","last_change":"2024-09-15T03:40:54.221919+0000","last_active":"2024-09-15T03:40:54.221970+0000","last_peered":"2024-09-15T03:40:54.221970+0000","last_clean":"2024-09-15T03:40:54.221970+0000","last_became_active":"2024-09-15T03:36:48.218083+0000","last_became_peered":"2024-09-15T03:36:48.218083+0000","last_unstale":"2024-09-15T03:40:54.221970+0000","last_undegraded":"2024-09-15T03:40:54.221970+0000","last_fullsized":"2024-09-15T03:40:54.221970+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'7969","last_scrub_stamp":"2024-09-15T03:40:54.221766+0000","last_deep_scrub":"24'7969","last_deep_scrub_stamp":"2024-09-15T03:40:54.221766+0000","last_clean_scrub_stamp":"2024-09-15T03:40:54.221766+0000","objects_scrubbed":25,"log_size":7969,"log_dups_size":0,"ondisk_log_size":7969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:57:37.082720+0000","scrub_duration":0.32031514999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663808,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":381,"num_read_kb":409,"num_write":5985,"num_write_kb":152253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"24'5871","reported_seq":5914,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:41:06.714875+0000","last_change":"2024-09-15T03:41:06.714838+0000","last_active":"2024-09-15T03:41:06.714875+0000","last_peered":"2024-09-15T03:41:06.714875+0000","last_clean":"2024-09-15T03:41:06.714875+0000","last_became_active":"2024-09-15T03:36:48.155263+0000","last_became_peered":"2024-09-15T03:36:48.155263+0000","last_unstale":"2024-09-15T03:41:06.714875+0000","last_undegraded":"2024-09-15T03:41:06.714875+0000","last_fullsized":"2024-09-15T03:41:06.714875+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'5871","last_scrub_stamp":"2024-09-15T03:41:06.714714+0000","last_deep_scrub":"24'5871","last_deep_scrub_stamp":"2024-09-15T03:41:06.714714+0000","last_clean_scrub_stamp":"2024-09-15T03:41:06.714714+0000","objects_scrubbed":34,"log_size":5871,"log_dups_size":0,"ondisk_log_size":5871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:14:56.633769+0000","scrub_duration":0.46467073599999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":137103712,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":68,"num_read_kb":57,"num_write":5841,"num_write_kb":205281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"24'3170","reported_seq":3192,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:41:02.654633+0000","last_change":"2024-09-15T03:41:02.654604+0000","last_active":"2024-09-15T03:41:02.654633+0000","last_peered":"2024-09-15T03:41:02.654633+0000","last_clean":"2024-09-15T03:41:02.654633+0000","last_became_active":"2024-09-15T03:36:48.137589+0000","last_became_peered":"2024-09-15T03:36:48.137589+0000","last_unstale":"2024-09-15T03:41:02.654633+0000","last_undegraded":"2024-09-15T03:41:02.654633+0000","last_fullsized":"2024-09-15T03:41:02.654633+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'3170","last_scrub_stamp":"2024-09-15T03:41:02.654523+0000","last_deep_scrub":"24'3170","last_deep_scrub_stamp":"2024-09-15T03:41:02.654523+0000","last_clean_scrub_stamp":"2024-09-15T03:41:02.654523+0000","objects_scrubbed":20,"log_size":3170,"log_dups_size":0,"ondisk_log_size":3170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T13:52:24.275895+0000","scrub_duration":0.156042232,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3149,"num_write_kb":152324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"24'4762","reported_seq":4802,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:41:04.789904+0000","last_change":"2024-09-15T03:41:04.789872+0000","last_active":"2024-09-15T03:41:04.789904+0000","last_peered":"2024-09-15T03:41:04.789904+0000","last_clean":"2024-09-15T03:41:04.789904+0000","last_became_active":"2024-09-15T03:36:48.151365+0000","last_became_peered":"2024-09-15T03:36:48.151365+0000","last_unstale":"2024-09-15T03:41:04.789904+0000","last_undegraded":"2024-09-15T03:41:04.789904+0000","last_fullsized":"2024-09-15T03:41:04.789904+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'4762","last_scrub_stamp":"2024-09-15T03:41:04.789781+0000","last_deep_scrub":"24'4762","last_deep_scrub_stamp":"2024-09-15T03:41:04.789781+0000","last_clean_scrub_stamp":"2024-09-15T03:41:04.789781+0000","objects_scrubbed":30,"log_size":4762,"log_dups_size":0,"ondisk_log_size":4762,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T12:58:01.226468+0000","scrub_duration":0.32063545500000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121635328,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":72,"num_read_kb":60,"num_write":4709,"num_write_kb":215880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"24'5518","reported_seq":5534,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:48.330010+0000","last_change":"2024-09-15T03:36:48.155332+0000","last_active":"2024-09-15T03:40:48.330010+0000","last_peered":"2024-09-15T03:40:48.330010+0000","last_clean":"2024-09-15T03:40:48.330010+0000","last_became_active":"2024-09-15T03:36:48.154955+0000","last_became_peered":"2024-09-15T03:36:48.154955+0000","last_unstale":"2024-09-15T03:40:48.330010+0000","last_undegraded":"2024-09-15T03:40:48.330010+0000","last_fullsized":"2024-09-15T03:40:48.330010+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":5518,"log_dups_size":0,"ondisk_log_size":5518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117440512,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":5499,"num_write_kb":248958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"24'4090","reported_seq":4196,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:48.344588+0000","last_change":"2024-09-15T03:36:48.162708+0000","last_active":"2024-09-15T03:40:48.344588+0000","last_peered":"2024-09-15T03:40:48.344588+0000","last_clean":"2024-09-15T03:40:48.344588+0000","last_became_active":"2024-09-15T03:36:48.162401+0000","last_became_peered":"2024-09-15T03:36:48.162401+0000","last_unstale":"2024-09-15T03:40:48.344588+0000","last_undegraded":"2024-09-15T03:40:48.344588+0000","last_fullsized":"2024-09-15T03:40:48.344588+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":4090,"log_dups_size":0,"ondisk_log_size":4090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104857600,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":310,"num_read_kb":242,"num_write":4350,"num_write_kb":177773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"24'3616","reported_seq":3647,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:48.333323+0000","last_change":"2024-09-15T03:36:48.162180+0000","last_active":"2024-09-15T03:40:48.333323+0000","last_peered":"2024-09-15T03:40:48.333323+0000","last_clean":"2024-09-15T03:40:48.333323+0000","last_became_active":"2024-09-15T03:36:48.161897+0000","last_became_peered":"2024-09-15T03:36:48.161897+0000","last_unstale":"2024-09-15T03:40:48.333323+0000","last_undegraded":"2024-09-15T03:40:48.333323+0000","last_fullsized":"2024-09-15T03:40:48.333323+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":0,"log_size":3616,"log_dups_size":0,"ondisk_log_size":3616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":130023424,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":75,"num_read_kb":60,"num_write":3605,"num_write_kb":185220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"24'4082","reported_seq":4105,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:41:10.535847+0000","last_change":"2024-09-15T03:41:10.535781+0000","last_active":"2024-09-15T03:41:10.535847+0000","last_peered":"2024-09-15T03:41:10.535847+0000","last_clean":"2024-09-15T03:41:10.535847+0000","last_became_active":"2024-09-15T03:36:48.149651+0000","last_became_peered":"2024-09-15T03:36:48.149651+0000","last_unstale":"2024-09-15T03:41:10.535847+0000","last_undegraded":"2024-09-15T03:41:10.535847+0000","last_fullsized":"2024-09-15T03:41:10.535847+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'4082","last_scrub_stamp":"2024-09-15T03:41:10.535652+0000","last_deep_scrub":"24'4082","last_deep_scrub_stamp":"2024-09-15T03:41:10.535652+0000","last_clean_scrub_stamp":"2024-09-15T03:41:10.535652+0000","objects_scrubbed":27,"log_size":4082,"log_dups_size":0,"ondisk_log_size":4082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:57:03.680831+0000","scrub_duration":0.33710412099999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":111869952,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4006,"num_write_kb":198851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"24'2817","reported_seq":2855,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:41:10.604994+0000","last_change":"2024-09-15T03:41:10.604946+0000","last_active":"2024-09-15T03:41:10.604994+0000","last_peered":"2024-09-15T03:41:10.604994+0000","last_clean":"2024-09-15T03:41:10.604994+0000","last_became_active":"2024-09-15T03:36:48.159380+0000","last_became_peered":"2024-09-15T03:36:48.159380+0000","last_unstale":"2024-09-15T03:41:10.604994+0000","last_undegraded":"2024-09-15T03:41:10.604994+0000","last_fullsized":"2024-09-15T03:41:10.604994+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'2817","last_scrub_stamp":"2024-09-15T03:41:10.604847+0000","last_deep_scrub":"24'2817","last_deep_scrub_stamp":"2024-09-15T03:41:10.604847+0000","last_clean_scrub_stamp":"2024-09-15T03:41:10.604847+0000","objects_scrubbed":20,"log_size":2817,"log_dups_size":0,"ondisk_log_size":2817,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:25:40.151554+0000","scrub_duration":0.28442361100000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":50,"num_read_kb":41,"num_write":2819,"num_write_kb":144138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"24'5748","reported_seq":5776,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:52.818187+0000","last_change":"2024-09-15T03:40:52.818147+0000","last_active":"2024-09-15T03:40:52.818187+0000","last_peered":"2024-09-15T03:40:52.818187+0000","last_clean":"2024-09-15T03:40:52.818187+0000","last_became_active":"2024-09-15T03:36:48.150784+0000","last_became_peered":"2024-09-15T03:36:48.150784+0000","last_unstale":"2024-09-15T03:40:52.818187+0000","last_undegraded":"2024-09-15T03:40:52.818187+0000","last_fullsized":"2024-09-15T03:40:52.818187+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'5748","last_scrub_stamp":"2024-09-15T03:40:52.818031+0000","last_deep_scrub":"24'5748","last_deep_scrub_stamp":"2024-09-15T03:40:52.818031+0000","last_clean_scrub_stamp":"2024-09-15T03:40:52.818031+0000","objects_scrubbed":34,"log_size":5748,"log_dups_size":0,"ondisk_log_size":5748,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:21.091184+0000","scrub_duration":0.37681035499999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":138543104,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":22,"num_read_kb":16,"num_write":5677,"num_write_kb":255624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"24'8378","reported_seq":8404,"reported_epoch":24,"state":"active+clean+scrubbing+deep","last_fresh":"2024-09-15T03:41:13.725030+0000","last_change":"2024-09-15T03:41:13.365291+0000","last_active":"2024-09-15T03:41:13.725030+0000","last_peered":"2024-09-15T03:41:13.725030+0000","last_clean":"2024-09-15T03:41:13.725030+0000","last_became_active":"2024-09-15T03:36:48.159470+0000","last_became_peered":"2024-09-15T03:36:48.159470+0000","last_unstale":"2024-09-15T03:41:13.725030+0000","last_undegraded":"2024-09-15T03:41:13.725030+0000","last_fullsized":"2024-09-15T03:41:13.725030+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:36:47.712904+0000","last_clean_scrub_stamp":"2024-09-15T03:36:47.712904+0000","objects_scrubbed":24,"log_size":8378,"log_dups_size":0,"ondisk_log_size":8378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 0s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":126026248,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":36,"num_read_kb":28,"num_write":8349,"num_write_kb":200158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":3570163960,"num_objects":878,"num_object_clones":0,"num_object_copies":1756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":878,"num_whiteouts":0,"num_read":2563,"num_read_kb":2579,"num_write":159019,"num_write_kb":5633737,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":6842679296,"data_stored":6838412408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161498,"ondisk_log_size":161498,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542202,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1361776,"kb_used_data":1047784,"kb_used_omap":1,"kb_used_meta":313982,"kb_avail":93010064,"statfs":{"total":96636764160,"available":95242305536,"internally_reserved":0,"allocated":1072930816,"data_stored":1072149589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":321517912},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 03:40:48 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":1.5880000000000001,"15min":2.444},"min":{"1min":0.42099999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.262,"5min":35.189,"15min":35.189},"last":1.054},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.77400000000000002,"15min":0.75},"min":{"1min":0.48999999999999999,"5min":0.245,"15min":0.245},"max":{"1min":1.2709999999999999,"5min":1.427,"15min":1.427},"last":0.55600000000000005}]},{"osd":1,"last update":"Sun Sep 15 03:40:48 2024","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":1.6299999999999999,"15min":2.4159999999999999},"min":{"1min":0.55700000000000005,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.1779999999999999,"5min":35.223999999999997,"15min":35.223999999999997},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.88,"5min":0.81100000000000005,"15min":0.77500000000000002},"min":{"1min":0.70399999999999996,"5min":0.308,"15min":0.308},"max":{"1min":1.1619999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.96299999999999997}]},{"osd":2,"last update":"Sun Sep 15 03:40:48 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.76500000000000001,"15min":0.73999999999999999},"min":{"1min":0.63600000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.196,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.86599999999999999},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.728,"15min":2.5150000000000001},"min":{"1min":0.79300000000000004,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.1990000000000001,"5min":35.283000000000001,"15min":35.283000000000001},"last":1.0089999999999999}]},{"osd":3,"last update":"Sun Sep 15 03:40:48 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":0.84499999999999997,"15min":0.76100000000000001},"min":{"1min":0.64700000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.3360000000000001,"5min":1.71,"15min":1.71},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.8080000000000001,"15min":2.5699999999999998},"min":{"1min":0.88800000000000001,"5min":0.378,"15min":0.378},"max":{"1min":1.369,"5min":35.255000000000003,"15min":35.255000000000003},"last":0.78500000000000003}]},{"osd":4,"last update":"Sun Sep 15 03:40:48 2024","interfaces":[{"interface":"back","average":{"1min":1.002,"5min":1.111,"15min":0.82899999999999996},"min":{"1min":0.69299999999999995,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.474,"5min":25.52,"15min":25.52},"last":0.50600000000000001},{"interface":"front","average":{"1min":1.024,"5min":0.871,"15min":0.79100000000000004},"min":{"1min":0.68899999999999995,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.2809999999999999,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.59199999999999997}]},{"osd":5,"last update":"Sun Sep 15 03:40:48 2024","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.8500000000000001,"15min":2.6480000000000001},"min":{"1min":0.78800000000000003,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.343,"5min":35.070999999999998,"15min":35.070999999999998},"last":0.83699999999999997},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.101,"15min":1.107},"min":{"1min":1.075,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.421,"5min":5.2999999999999998,"15min":5.2999999999999998},"last":0.81999999999999995}]},{"osd":6,"last update":"Sun Sep 15 03:40:48 2024","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.024,"15min":0.90400000000000003},"min":{"1min":1.085,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.429,"5min":2.5449999999999999,"15min":2.5449999999999999},"last":0.755},{"interface":"front","average":{"1min":1.151,"5min":0.97299999999999998,"15min":0.86899999999999999},"min":{"1min":0.71499999999999997,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.452,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":0.79400000000000004}]}]},{"osd":6,"up_from":14,"seq":60129542201,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":651348,"kb_used_data":349836,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93720492,"statfs":{"total":96636764160,"available":95969783808,"internally_reserved":0,"allocated":358232064,"data_stored":357835405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 03:40:43 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":1.0049999999999999,"15min":1.155},"min":{"1min":0.253,"5min":0.25,"15min":0.25},"max":{"1min":1.208,"5min":7.3170000000000002,"15min":7.3170000000000002},"last":1.0029999999999999},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.82399999999999995,"15min":0.77300000000000002},"min":{"1min":0.47799999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.212,"5min":7.3849999999999998,"15min":7.3849999999999998},"last":0.92300000000000004}]},{"osd":1,"last update":"Sun Sep 15 03:40:43 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.72899999999999998,"15min":0.69099999999999995},"min":{"1min":0.30299999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.157,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.89600000000000002,"15min":0.91800000000000004},"min":{"1min":0.45800000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.304,"5min":7.3559999999999999,"15min":7.3559999999999999},"last":0.97299999999999998}]},{"osd":2,"last update":"Sun Sep 15 03:40:43 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.92800000000000005,"15min":0.93500000000000005},"min":{"1min":0.436,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.286,"5min":7.4960000000000004,"15min":7.4960000000000004},"last":1.03},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.84699999999999998,"15min":0.88600000000000001},"min":{"1min":0.49199999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.1499999999999999,"5min":1.575,"15min":1.575},"last":1.1439999999999999}]},{"osd":3,"last update":"Sun Sep 15 03:40:43 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":1.0569999999999999,"15min":1.1499999999999999},"min":{"1min":0.42399999999999999,"5min":0.308,"15min":0.308},"max":{"1min":1.236,"5min":7.5869999999999997,"15min":7.5869999999999997},"last":0.86699999999999999},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.2869999999999999,"15min":1.526},"min":{"1min":0.53800000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.423,"5min":11.494,"15min":11.494},"last":1.169}]},{"osd":4,"last update":"Sun Sep 15 03:40:43 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":1.0449999999999999,"15min":1.0820000000000001},"min":{"1min":0.32600000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.319,"5min":10.348000000000001,"15min":10.348000000000001},"last":0.94499999999999995},{"interface":"front","average":{"1min":1.083,"5min":1.0509999999999999,"15min":1.0049999999999999},"min":{"1min":0.44,"5min":0.316,"15min":0.316},"max":{"1min":1.373,"5min":7.5620000000000003,"15min":7.5620000000000003},"last":0.82499999999999996}]},{"osd":5,"last update":"Sun Sep 15 03:40:43 2024","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.86899999999999999,"15min":0.83799999999999997},"min":{"1min":0.54500000000000004,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.3400000000000001,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":1.147,"5min":1.0149999999999999,"15min":0.96699999999999997},"min":{"1min":0.56699999999999995,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.444,"5min":1.623,"15min":1.623},"last":1.0660000000000001}]},{"osd":7,"last update":"Sun Sep 15 03:40:43 2024","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.121,"15min":1.0149999999999999},"min":{"1min":0.40500000000000003,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.4279999999999999,"5min":10.923999999999999,"15min":10.923999999999999},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.1160000000000001,"15min":1.01},"min":{"1min":0.52100000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.7130000000000001,"5min":10.74,"15min":10.74},"last":1.1020000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542201,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1179932,"kb_used_data":874260,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":93191908,"statfs":{"total":96636764160,"available":95428513792,"internally_reserved":0,"allocated":895242240,"data_stored":894489264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 03:40:46 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.69699999999999995,"15min":0.66300000000000003},"min":{"1min":0.316,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.0049999999999999,"5min":1.3,"15min":1.3},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.80600000000000005,"15min":0.83899999999999997},"min":{"1min":0.36099999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.3300000000000001,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":1.077}]},{"osd":1,"last update":"Sun Sep 15 03:40:46 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.78200000000000003,"15min":0.71599999999999997},"min":{"1min":0.33200000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.2450000000000001,"5min":2.302,"15min":2.302},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.73399999999999999,"15min":0.69599999999999995},"min":{"1min":0.36099999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.222,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.81299999999999994}]},{"osd":2,"last update":"Sun Sep 15 03:40:46 2024","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.85399999999999998,"15min":0.83499999999999996},"min":{"1min":0.48199999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":2.5779999999999998,"5min":2.5779999999999998,"15min":2.5779999999999998},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.85499999999999998,"15min":0.85999999999999999},"min":{"1min":0.376,"5min":0.314,"15min":0.314},"max":{"1min":1.399,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.87}]},{"osd":3,"last update":"Sun Sep 15 03:40:46 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":1.401,"15min":1.9419999999999999},"min":{"1min":0.438,"5min":0.375,"15min":0.375},"max":{"1min":1.2789999999999999,"5min":26.664999999999999,"15min":26.664999999999999},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.96999999999999997,"5min":1.538,"15min":2.1230000000000002},"min":{"1min":0.40500000000000003,"5min":0.312,"15min":0.312},"max":{"1min":1.327,"5min":26.748000000000001,"15min":26.748000000000001},"last":0.95599999999999996}]},{"osd":4,"last update":"Sun Sep 15 03:40:46 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.90600000000000003,"15min":0.88200000000000001},"min":{"1min":0.40600000000000003,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.3280000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.92800000000000005,"15min":0.88200000000000001},"min":{"1min":0.51100000000000001,"5min":0.318,"15min":0.318},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":0.98799999999999999}]},{"osd":6,"last update":"Sun Sep 15 03:40:46 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.97699999999999998,"15min":0.96099999999999997},"min":{"1min":0.52900000000000003,"5min":0.373,"15min":0.373},"max":{"1min":1.3320000000000001,"5min":1.9730000000000001,"15min":1.9730000000000001},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.056,"5min":1.04,"15min":1.004},"min":{"1min":0.436,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.452,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":0.93500000000000005}]},{"osd":7,"last update":"Sun Sep 15 03:40:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":0.97999999999999998,"15min":0.91400000000000003},"min":{"1min":0.47899999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":1.4450000000000001},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.121,"15min":1.077},"min":{"1min":0.55500000000000005,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.6240000000000001,"5min":4.3070000000000004,"15min":4.3070000000000004},"last":1.393}]}]},{"osd":3,"up_from":14,"seq":60129542201,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1279280,"kb_used_data":981928,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93092560,"statfs":{"total":96636764160,"available":95326781440,"internally_reserved":0,"allocated":1005494272,"data_stored":1004659645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 03:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.63900000000000001,"15min":0.69699999999999995},"min":{"1min":0.28199999999999997,"5min":0.22,"15min":0.22},"max":{"1min":0.93200000000000005,"5min":4.7809999999999997,"15min":4.7809999999999997},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.79300000000000004,"15min":0.96899999999999997},"min":{"1min":0.29199999999999998,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":1.2609999999999999,"5min":7.7370000000000001,"15min":7.7370000000000001},"last":1.0629999999999999}]},{"osd":1,"last update":"Sun Sep 15 03:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.77900000000000003,"15min":0.63800000000000001},"min":{"1min":0.29099999999999998,"5min":0.217,"15min":0.217},"max":{"1min":1.032,"5min":13.749000000000001,"15min":13.749000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.83499999999999996,"5min":1.296,"15min":0.96799999999999997},"min":{"1min":0.44500000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.2989999999999999,"5min":46.225999999999999,"15min":46.225999999999999},"last":0.68200000000000005}]},{"osd":2,"last update":"Sun Sep 15 03:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.75700000000000001,"15min":0.71299999999999997},"min":{"1min":0.42399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.88,"5min":0.76200000000000001,"15min":0.72599999999999998},"min":{"1min":0.46899999999999997,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.167,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.94799999999999995}]},{"osd":4,"last update":"Sun Sep 15 03:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.93100000000000005,"15min":0.94299999999999995},"min":{"1min":0.56299999999999994,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.5660000000000001,"5min":2.9870000000000001,"15min":2.9870000000000001},"last":0.745},{"interface":"front","average":{"1min":1.016,"5min":0.95099999999999996,"15min":0.96799999999999997},"min":{"1min":0.38500000000000001,"5min":0.376,"15min":0.376},"max":{"1min":1.3380000000000001,"5min":2.609,"15min":2.609},"last":0.81200000000000006}]},{"osd":5,"last update":"Sun Sep 15 03:40:45 2024","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":0.90900000000000003,"15min":0.85499999999999998},"min":{"1min":0.42499999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.6140000000000001,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":1.0289999999999999},{"interface":"front","average":{"1min":1.155,"5min":1.071,"15min":1.1060000000000001},"min":{"1min":0.498,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.6679999999999999,"5min":5.0309999999999997,"15min":5.0309999999999997},"last":1.006}]},{"osd":6,"last update":"Sun Sep 15 03:40:45 2024","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.014,"15min":0.98599999999999999},"min":{"1min":0.59899999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":1.111},{"interface":"front","average":{"1min":1.091,"5min":1.002,"15min":0.97199999999999998},"min":{"1min":0.61099999999999999,"5min":0.503,"15min":0.503},"max":{"1min":1.476,"5min":2.5550000000000002,"15min":2.5550000000000002},"last":0.98199999999999998}]},{"osd":7,"last update":"Sun Sep 15 03:40:45 2024","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.129,"15min":1.1830000000000001},"min":{"1min":0.74099999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.587,"5min":5.5439999999999996,"15min":5.5439999999999996},"last":0.90700000000000003},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.2989999999999999,"15min":1.5269999999999999},"min":{"1min":0.71899999999999997,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.5529999999999999,"5min":9.5850000000000009,"15min":9.5850000000000009},"last":1.149}]}]},{"osd":2,"up_from":14,"seq":60129542202,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":687248,"kb_used_data":377416,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":93684592,"statfs":{"total":96636764160,"available":95933022208,"internally_reserved":0,"allocated":386473984,"data_stored":386097365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 03:40:46 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.59699999999999998,"15min":0.61199999999999999},"min":{"1min":0.39800000000000002,"5min":0.193,"15min":0.193},"max":{"1min":0.96499999999999997,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.66700000000000004,"15min":0.65700000000000003},"min":{"1min":0.54000000000000004,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":1.228,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.78300000000000003}]},{"osd":1,"last update":"Sun Sep 15 03:40:46 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.73099999999999998,"15min":0.72599999999999998},"min":{"1min":0.47299999999999998,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.044,"5min":1.258,"15min":1.258},"last":0.999},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.61599999999999999,"15min":0.57999999999999996},"min":{"1min":0.41999999999999998,"5min":0.222,"15min":0.222},"max":{"1min":0.92800000000000005,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.65200000000000002}]},{"osd":3,"last update":"Sun Sep 15 03:40:46 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.68300000000000005,"15min":0.66000000000000003},"min":{"1min":0.42399999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.98299999999999998,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.77800000000000002,"15min":0.73399999999999999},"min":{"1min":0.40100000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.85899999999999999}]},{"osd":4,"last update":"Sun Sep 15 03:40:46 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.84099999999999997,"15min":0.77900000000000003},"min":{"1min":0.55500000000000005,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":0.82699999999999996},{"interface":"front","average":{"1min":1.02,"5min":0.91100000000000003,"15min":0.86599999999999999},"min":{"1min":0.53100000000000003,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.4239999999999999,"5min":1.573,"15min":1.573},"last":1.0580000000000001}]},{"osd":5,"last update":"Sun Sep 15 03:40:46 2024","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.94299999999999995,"15min":0.95399999999999996},"min":{"1min":0.68500000000000005,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.3680000000000001,"5min":4.5839999999999996,"15min":4.5839999999999996},"last":1.236},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0549999999999999,"15min":1.113},"min":{"1min":0.50900000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.341,"5min":4.8230000000000004,"15min":4.8230000000000004},"last":0.88500000000000001}]},{"osd":6,"last update":"Sun Sep 15 03:40:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":0.94199999999999995,"15min":0.86799999999999999},"min":{"1min":0.81100000000000005,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.288,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":1.157},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.03,"15min":0.96999999999999997},"min":{"1min":0.49299999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.425,"5min":1.9039999999999999,"15min":1.9039999999999999},"last":1.2749999999999999}]},{"osd":7,"last update":"Sun Sep 15 03:40:46 2024","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.0389999999999999,"15min":1.044},"min":{"1min":0.72599999999999998,"5min":0.379,"15min":0.379},"max":{"1min":1.4590000000000001,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.169,"5min":1.3540000000000001,"15min":1.6080000000000001},"min":{"1min":0.74099999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.4770000000000001,"5min":10.428000000000001,"15min":10.428000000000001},"last":1.335}]}]},{"osd":4,"up_from":13,"seq":55834574906,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1618136,"kb_used_data":1320784,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":92753704,"statfs":{"total":96636764160,"available":94979792896,"internally_reserved":0,"allocated":1352482816,"data_stored":1351527913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 03:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.85799999999999998,"15min":0.93999999999999995},"min":{"1min":0.311,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.137,"5min":6.5010000000000003,"15min":6.5010000000000003},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.85299999999999998,"15min":0.89300000000000002},"min":{"1min":0.34300000000000003,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.159,"5min":3.645,"15min":3.645},"last":0.67700000000000005}]},{"osd":1,"last update":"Sun Sep 15 03:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.84499999999999997,"15min":0.88900000000000001},"min":{"1min":0.29299999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.2589999999999999,"5min":3.7069999999999999,"15min":3.7069999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.81699999999999995,"15min":0.80200000000000005},"min":{"1min":0.44600000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.272,"5min":2.5779999999999998,"15min":2.5779999999999998},"last":1.0469999999999999}]},{"osd":2,"last update":"Sun Sep 15 03:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.84999999999999998,"15min":0.81100000000000005},"min":{"1min":0.45000000000000001,"5min":0.311,"15min":0.311},"max":{"1min":1.212,"5min":2.766,"15min":2.766},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.84199999999999997,"15min":0.77200000000000002},"min":{"1min":0.375,"5min":0.316,"15min":0.316},"max":{"1min":1.256,"5min":3.2749999999999999,"15min":3.2749999999999999},"last":0.92800000000000005}]},{"osd":3,"last update":"Sun Sep 15 03:40:45 2024","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.0609999999999999,"15min":1.173},"min":{"1min":0.501,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.357,"5min":6.3620000000000001,"15min":6.3620000000000001},"last":0.78800000000000003},{"interface":"front","average":{"1min":1.0640000000000001,"5min":0.94899999999999995,"15min":0.87},"min":{"1min":0.58799999999999997,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.4330000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":1.153}]},{"osd":5,"last update":"Sun Sep 15 03:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":1.0069999999999999,"15min":1.0629999999999999},"min":{"1min":0.41899999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.6890000000000001,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.171,"5min":1.143,"15min":1.107},"min":{"1min":0.53000000000000003,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.651,"5min":4.1710000000000003,"15min":4.1710000000000003},"last":1.248}]},{"osd":6,"last update":"Sun Sep 15 03:40:45 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.0309999999999999,"15min":1.0389999999999999},"min":{"1min":0.45800000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.6120000000000001,"5min":3.669,"15min":3.669},"last":1.115},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.0609999999999999,"15min":0.96399999999999997},"min":{"1min":0.73799999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.581,"5min":2.5329999999999999,"15min":2.5329999999999999},"last":1.2030000000000001}]},{"osd":7,"last update":"Sun Sep 15 03:40:45 2024","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.018,"15min":0.92700000000000005},"min":{"1min":0.56599999999999995,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.498,"5min":3.238,"15min":3.238},"last":1.23},{"interface":"front","average":{"1min":1.244,"5min":1.1299999999999999,"15min":1.109},"min":{"1min":0.68700000000000006,"5min":0.378,"15min":0.378},"max":{"1min":1.552,"5min":3.8260000000000001,"15min":3.8260000000000001},"last":1.286}]}]},{"osd":1,"up_from":13,"seq":55834574905,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":986296,"kb_used_data":676464,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":93385544,"statfs":{"total":96636764160,"available":95626797056,"internally_reserved":0,"allocated":692699136,"data_stored":692171708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 03:40:47 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.58699999999999997,"15min":0.55100000000000005},"min":{"1min":0.27600000000000002,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.874,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.65900000000000003,"15min":0.60499999999999998},"min":{"1min":0.22900000000000001,"5min":0.187,"15min":0.187},"max":{"1min":0.91000000000000003,"5min":6.6050000000000004,"15min":6.6050000000000004},"last":0.88200000000000001}]},{"osd":2,"last update":"Sun Sep 15 03:40:47 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.64100000000000001,"15min":0.63100000000000001},"min":{"1min":0.314,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.88700000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.77900000000000003,"15min":0.751},"min":{"1min":0.32000000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.228,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":0.69299999999999995}]},{"osd":3,"last update":"Sun Sep 15 03:40:47 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.70599999999999996,"15min":0.64600000000000002},"min":{"1min":0.373,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.85599999999999998,"15min":0.74199999999999999},"min":{"1min":0.40300000000000002,"5min":0.255,"15min":0.255},"max":{"1min":1.1659999999999999,"5min":5.6950000000000003,"15min":5.6950000000000003},"last":0.84199999999999997}]},{"osd":4,"last update":"Sun Sep 15 03:40:47 2024","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.0409999999999999,"15min":0.90400000000000003},"min":{"1min":0.46000000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.3959999999999999,"5min":9.1080000000000005,"15min":9.1080000000000005},"last":1.069},{"interface":"front","average":{"1min":1.149,"5min":1.1120000000000001,"15min":0.98799999999999999},"min":{"1min":0.59299999999999997,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.5660000000000001,"5min":9.1440000000000001,"15min":9.1440000000000001},"last":1.157}]},{"osd":5,"last update":"Sun Sep 15 03:40:47 2024","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":0.93799999999999994,"15min":0.90100000000000002},"min":{"1min":0.47499999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.5349999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.995},{"interface":"front","average":{"1min":1.151,"5min":1.0289999999999999,"15min":0.97499999999999998},"min":{"1min":0.49099999999999999,"5min":0.314,"15min":0.314},"max":{"1min":1.4690000000000001,"5min":2.3580000000000001,"15min":2.3580000000000001},"last":0.96299999999999997}]},{"osd":6,"last update":"Sun Sep 15 03:40:47 2024","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.0029999999999999,"15min":0.94099999999999995},"min":{"1min":0.436,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.387,"5min":6.4109999999999996,"15min":6.4109999999999996},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.087,"5min":1.0489999999999999,"15min":1.006},"min":{"1min":0.53600000000000003,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.5229999999999999,"5min":5.3170000000000002,"15min":5.3170000000000002},"last":1.1240000000000001}]},{"osd":7,"last update":"Sun Sep 15 03:40:47 2024","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.268,"15min":1.4279999999999999},"min":{"1min":0.55500000000000005,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.522,"5min":12.821,"15min":12.821},"last":1.0960000000000001},{"interface":"front","average":{"1min":1.137,"5min":1.105,"15min":1.073},"min":{"1min":0.59099999999999997,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.498,"5min":5.2530000000000001,"15min":5.2530000000000001},"last":1.216}]}]},{"osd":0,"up_from":13,"seq":55834574906,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1362504,"kb_used_data":1056832,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":93009336,"statfs":{"total":96636764160,"available":95241560064,"internally_reserved":0,"allocated":1082195968,"data_stored":1081279613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Sep 15 03:40:41 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.59399999999999997,"15min":0.59999999999999998},"min":{"1min":0.20799999999999999,"5min":0.19,"15min":0.19},"max":{"1min":2.9279999999999999,"5min":2.9279999999999999,"15min":2.9279999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.58099999999999996,"15min":0.54400000000000004},"min":{"1min":0.41399999999999998,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.68000000000000005}]},{"osd":2,"last update":"Sun Sep 15 03:40:47 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.61899999999999999,"15min":0.55900000000000005},"min":{"1min":0.30399999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.69499999999999995,"15min":0.69899999999999995},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.1479999999999999,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":0.81399999999999995}]},{"osd":3,"last update":"Sun Sep 15 03:40:47 2024","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.73699999999999999,"15min":0.72999999999999998},"min":{"1min":0.47499999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":2.8860000000000001,"5min":2.8860000000000001,"15min":2.8860000000000001},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.69799999999999995,"15min":0.65800000000000003},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.871}]},{"osd":4,"last update":"Sun Sep 15 03:40:41 2024","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.83099999999999996,"15min":0.80200000000000005},"min":{"1min":0.45900000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.1919999999999999,"5min":2.403,"15min":2.403},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.135,"5min":0.89700000000000002,"15min":0.83499999999999996},"min":{"1min":0.41299999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":3.1059999999999999,"5min":3.1059999999999999,"15min":3.1059999999999999},"last":0.92700000000000005}]},{"osd":5,"last update":"Sun Sep 15 03:40:47 2024","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":0.93000000000000005,"15min":0.84399999999999997},"min":{"1min":0.68899999999999995,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":2.9609999999999999,"5min":2.9609999999999999,"15min":2.9609999999999999},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.1000000000000001,"5min":0.96599999999999997,"15min":0.88400000000000001},"min":{"1min":0.51200000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.54,"5min":2.4319999999999999,"15min":2.4319999999999999},"last":1.3029999999999999}]},{"osd":6,"last update":"Sun Sep 15 03:40:47 2024","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":0.92100000000000004,"15min":0.871},"min":{"1min":0.67500000000000004,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.3580000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.0329999999999999,"15min":1.0089999999999999},"min":{"1min":0.56899999999999995,"5min":0.442,"15min":0.442},"max":{"1min":1.639,"5min":2.5459999999999998,"15min":2.5459999999999998},"last":1.0449999999999999}]},{"osd":7,"last update":"Sun Sep 15 03:40:47 2024","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.206,"15min":1.2989999999999999},"min":{"1min":0.54300000000000004,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.466,"5min":10.561999999999999,"15min":10.561999999999999},"last":1.3520000000000001},{"interface":"front","average":{"1min":1.254,"5min":0.98899999999999999,"15min":0.91100000000000003},"min":{"1min":0.64200000000000002,"5min":0.377,"15min":0.377},"max":{"1min":3.2120000000000002,"5min":3.2120000000000002,"15min":3.2120000000000002},"last":1.083}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1081413632,"data_stored":1080658160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":692445184,"data_stored":692079164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":386224128,"data_stored":386004840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1005244416,"data_stored":1004567120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1352232960,"data_stored":1351435388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":894988288,"data_stored":894396720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":357982208,"data_stored":357742880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1072148480,"data_stored":1071528136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T03:41:15.813 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-15T03:36:47.712904+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=36, tm_sec=47, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=40, tm_sec=50, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T03:41:15.813 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-15T03:36:47.712904+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=36, tm_sec=47, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=40, tm_sec=50, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T03:41:15.813 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-15T03:36:47.712904+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=36, tm_sec=47, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=40, tm_sec=50, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T03:41:15.813 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-15T03:36:47.712904+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=36, tm_sec=47, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=40, tm_sec=50, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T03:41:15.814 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-15T03:36:47.712904+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=36, tm_sec=47, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=40, tm_sec=50, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T03:41:15.815 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-15T03:36:47.712904+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=36, tm_sec=47, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=40, tm_sec=50, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T03:41:15.815 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-15T03:41:35.816 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T03:41:35.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:35.926+0000 7fc0ae6fa700 1 Processor -- start 2024-09-15T03:41:35.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:35.926+0000 7fc0ae6fa700 1 -- start start 2024-09-15T03:41:35.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:35.926+0000 7fc0ae6fa700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fc0a80b2720 0x7fc0a8141810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:41:35.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:35.926+0000 7fc0ae6fa700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fc0a80afe00 0x7fc0a80ae220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:41:35.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:35.926+0000 7fc0ae6fa700 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0a80af260 con 0x7fc0a813ddf0 2024-09-15T03:41:35.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:35.926+0000 7fc0ae6fa700 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0a80b2af0 con 0x7fc0a80afe00 2024-09-15T03:41:35.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:35.926+0000 7fc0ae6fa700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0a80ac680 con 0x7fc0a80b2720 2024-09-15T03:41:35.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:35.926+0000 7fc0adef9700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7fc0a80afe00 0x7fc0a80ae220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:41:35.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:35.926+0000 7fc0acef7700 1 --1- >> v1:172.21.15.110:6789/0 conn(0x7fc0a813ddf0 0x7fc0a80af6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.110:6789/0 says I am v1:172.21.15.110:32966/0 (socket says 172.21.15.110:32966) 2024-09-15T03:41:35.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:35.926+0000 7fc0acef7700 1 -- 172.21.15.110:0/3905816861 learned_addr learned my addr 172.21.15.110:0/3905816861 (peer_addr_for_me v1:172.21.15.110:0/0) 2024-09-15T03:41:35.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:35.926+0000 7fc0ad6f8700 1 --2- 172.21.15.110:0/3905816861 >> v2:172.21.15.177:3300/0 conn(0x7fc0a80b2720 0x7fc0a8141810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:41:35.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:35.926+0000 7fc0adef9700 1 -- 172.21.15.110:0/3905816861 >> v2:172.21.15.177:3300/0 conn(0x7fc0a80b2720 msgr2=0x7fc0a8141810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:41:35.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:35.926+0000 7fc0adef9700 1 --2- 172.21.15.110:0/3905816861 >> v2:172.21.15.177:3300/0 conn(0x7fc0a80b2720 0x7fc0a8141810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:41:35.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:35.926+0000 7fc0adef9700 1 -- 172.21.15.110:0/3905816861 >> v1:172.21.15.110:6789/0 conn(0x7fc0a813ddf0 legacy=0x7fc0a80af6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:41:35.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:35.926+0000 7fc0adef9700 1 -- 172.21.15.110:0/3905816861 --> v2:172.21.15.110:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0a8141d50 con 0x7fc0a80afe00 2024-09-15T03:41:35.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:35.926+0000 7fc0adef9700 1 --2- 172.21.15.110:0/3905816861 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0a80afe00 0x7fc0a80ae220 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc0a806fdd0 tx=0x7fc0a0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30b90f4a48524050 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:41:35.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:35.926+0000 7fc09ffff700 1 -- 172.21.15.110:0/3905816861 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a000e070 con 0x7fc0a80afe00 2024-09-15T03:41:35.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:35.926+0000 7fc09ffff700 1 -- 172.21.15.110:0/3905816861 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0a000bc30 con 0x7fc0a80afe00 2024-09-15T03:41:35.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:35.926+0000 7fc09ffff700 1 -- 172.21.15.110:0/3905816861 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a0012430 con 0x7fc0a80afe00 2024-09-15T03:41:35.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:35.930+0000 7fc0ae6fa700 1 -- 172.21.15.110:0/3905816861 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0a80afe00 msgr2=0x7fc0a80ae220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:41:35.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:35.930+0000 7fc0ae6fa700 1 --2- 172.21.15.110:0/3905816861 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0a80afe00 0x7fc0a80ae220 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc0a806fdd0 tx=0x7fc0a0009780 comp rx=0 tx=0).stop 2024-09-15T03:41:35.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:35.930+0000 7fc0ae6fa700 1 -- 172.21.15.110:0/3905816861 shutdown_connections 2024-09-15T03:41:35.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:35.930+0000 7fc0ae6fa700 1 --2- 172.21.15.110:0/3905816861 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0a80afe00 0x7fc0a80ae220 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:41:35.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:35.930+0000 7fc0ae6fa700 1 --2- 172.21.15.110:0/3905816861 >> v2:172.21.15.177:3300/0 conn(0x7fc0a80b2720 0x7fc0a8141810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:41:35.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:35.930+0000 7fc0ae6fa700 1 -- 172.21.15.110:0/3905816861 >> 172.21.15.110:0/3905816861 conn(0x7fc0a8091190 msgr2=0x7fc0a8091590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:41:35.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:35.930+0000 7fc0ae6fa700 1 -- 172.21.15.110:0/3905816861 shutdown_connections 2024-09-15T03:41:35.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:35.930+0000 7fc0ae6fa700 1 -- 172.21.15.110:0/3905816861 wait complete. 2024-09-15T03:41:35.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:35.930+0000 7fc0ae6fa700 1 Processor -- start 2024-09-15T03:41:35.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:35.930+0000 7fc0ae6fa700 1 -- start start 2024-09-15T03:41:35.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:35.930+0000 7fc0ae6fa700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0a80afe00 0x7fc0a8144e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:41:35.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:35.930+0000 7fc0ae6fa700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fc0a80b2720 0x7fc0a8142620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:41:35.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:35.930+0000 7fc0ae6fa700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0a813ddf0 0x7fc0a8142b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:41:35.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:35.930+0000 7fc0ae6fa700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a814d9f0 con 0x7fc0a80afe00 2024-09-15T03:41:35.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:35.930+0000 7fc0ae6fa700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a8094820 con 0x7fc0a813ddf0 2024-09-15T03:41:35.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:35.930+0000 7fc0ae6fa700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a80955d0 con 0x7fc0a80b2720 2024-09-15T03:41:35.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:35.930+0000 7fc0ad6f8700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0a80afe00 0x7fc0a8144e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:41:35.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:35.930+0000 7fc0acef7700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fc0a80b2720 0x7fc0a8142620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:41:35.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:35.930+0000 7fc0ad6f8700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0a80afe00 0x7fc0a8144e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37422/0 (socket says 172.21.15.110:37422) 2024-09-15T03:41:35.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:35.930+0000 7fc0ad6f8700 1 -- 172.21.15.110:0/910280401 learned_addr learned my addr 172.21.15.110:0/910280401 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:41:35.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:35.930+0000 7fc0adef9700 1 --2- 172.21.15.110:0/910280401 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0a813ddf0 0x7fc0a8142b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:41:35.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:35.934+0000 7fc0acef7700 1 -- 172.21.15.110:0/910280401 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0a813ddf0 msgr2=0x7fc0a8142b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:41:35.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:35.934+0000 7fc0acef7700 1 --2- 172.21.15.110:0/910280401 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0a813ddf0 0x7fc0a8142b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:41:35.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:35.934+0000 7fc0acef7700 1 -- 172.21.15.110:0/910280401 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0a80afe00 msgr2=0x7fc0a8144e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:41:35.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:35.934+0000 7fc0acef7700 1 --2- 172.21.15.110:0/910280401 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0a80afe00 0x7fc0a8144e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:41:35.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:35.934+0000 7fc0acef7700 1 -- 172.21.15.110:0/910280401 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0a0009a20 con 0x7fc0a80b2720 2024-09-15T03:41:35.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:35.934+0000 7fc0acef7700 1 --2- 172.21.15.110:0/910280401 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fc0a80b2720 0x7fc0a8142620 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc0a400c4a0 tx=0x7fc0a4007f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:41:35.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:35.934+0000 7fc09d7fa700 1 -- 172.21.15.110:0/910280401 <== mon.2 v2:172.21.15.110:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a400ce50 con 0x7fc0a80b2720 2024-09-15T03:41:35.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:35.934+0000 7fc09d7fa700 1 -- 172.21.15.110:0/910280401 <== mon.2 v2:172.21.15.110:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0a4005b30 con 0x7fc0a80b2720 2024-09-15T03:41:35.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:35.934+0000 7fc0ae6fa700 1 -- 172.21.15.110:0/910280401 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0a81430a0 con 0x7fc0a80b2720 2024-09-15T03:41:35.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:35.934+0000 7fc09d7fa700 1 -- 172.21.15.110:0/910280401 <== mon.2 v2:172.21.15.110:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a40070d0 con 0x7fc0a80b2720 2024-09-15T03:41:35.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:35.934+0000 7fc0ae6fa700 1 -- 172.21.15.110:0/910280401 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0a8144050 con 0x7fc0a80b2720 2024-09-15T03:41:35.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:35.934+0000 7fc0ae6fa700 1 -- 172.21.15.110:0/910280401 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0a8141210 con 0x7fc0a80b2720 2024-09-15T03:41:35.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:35.934+0000 7fc09d7fa700 1 -- 172.21.15.110:0/910280401 <== mon.2 v2:172.21.15.110:3301/0 4 ==== mgrmap(e 5) v1 ==== 85847+0+0 (secure 0 0 0) 0x7fc0a40074c0 con 0x7fc0a80b2720 2024-09-15T03:41:35.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:35.938+0000 7fc09d7fa700 1 --2- 172.21.15.110:0/910280401 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fc08c067a80 0x7fc08c069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:41:35.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:35.938+0000 7fc09d7fa700 1 -- 172.21.15.110:0/910280401 <== mon.2 v2:172.21.15.110:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5641+0+0 (secure 0 0 0) 0x7fc0a4084950 con 0x7fc0a80b2720 2024-09-15T03:41:35.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:35.938+0000 7fc0ad6f8700 1 --2- 172.21.15.110:0/910280401 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fc08c067a80 0x7fc08c069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:41:35.943 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:35.938+0000 7fc0ad6f8700 1 --2- 172.21.15.110:0/910280401 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fc08c067a80 0x7fc08c069f30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc098006fd0 tx=0x7fc098008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:41:35.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:35.942+0000 7fc09d7fa700 1 -- 172.21.15.110:0/910280401 <== mon.2 v2:172.21.15.110:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fc0a8141210 con 0x7fc0a80b2720 2024-09-15T03:41:36.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.098+0000 7fc0ae6fa700 1 -- 172.21.15.110:0/910280401 --> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc0a80949e0 con 0x7fc08c067a80 2024-09-15T03:41:36.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.110+0000 7fc09d7fa700 1 -- 172.21.15.110:0/910280401 <== mgr.4104 v2:172.21.15.110:6832/22190 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+154354 (secure 0 0 0) 0x7fc0a80949e0 con 0x7fc08c067a80 2024-09-15T03:41:36.118 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-15T03:41:36.120 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.114+0000 7fc0ae6fa700 1 -- 172.21.15.110:0/910280401 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fc08c067a80 msgr2=0x7fc08c069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:41:36.120 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.114+0000 7fc0ae6fa700 1 --2- 172.21.15.110:0/910280401 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fc08c067a80 0x7fc08c069f30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc098006fd0 tx=0x7fc098008040 comp rx=0 tx=0).stop 2024-09-15T03:41:36.120 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.114+0000 7fc0ae6fa700 1 -- 172.21.15.110:0/910280401 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fc0a80b2720 msgr2=0x7fc0a8142620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:41:36.120 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.114+0000 7fc0ae6fa700 1 --2- 172.21.15.110:0/910280401 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fc0a80b2720 0x7fc0a8142620 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc0a400c4a0 tx=0x7fc0a4007f10 comp rx=0 tx=0).stop 2024-09-15T03:41:36.120 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.114+0000 7fc0ae6fa700 1 -- 172.21.15.110:0/910280401 shutdown_connections 2024-09-15T03:41:36.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.114+0000 7fc0ae6fa700 1 --2- 172.21.15.110:0/910280401 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7fc08c067a80 0x7fc08c069f30 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:41:36.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.114+0000 7fc0ae6fa700 1 --2- 172.21.15.110:0/910280401 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0a813ddf0 0x7fc0a8142b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:41:36.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.114+0000 7fc0ae6fa700 1 --2- 172.21.15.110:0/910280401 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7fc0a80b2720 0x7fc0a8142620 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:41:36.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.114+0000 7fc0ae6fa700 1 --2- 172.21.15.110:0/910280401 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0a80afe00 0x7fc0a8144e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:41:36.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.114+0000 7fc0ae6fa700 1 -- 172.21.15.110:0/910280401 >> 172.21.15.110:0/910280401 conn(0x7fc0a8091190 msgr2=0x7fc0a80b10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:41:36.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.114+0000 7fc0ae6fa700 1 -- 172.21.15.110:0/910280401 shutdown_connections 2024-09-15T03:41:36.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.114+0000 7fc0ae6fa700 1 -- 172.21.15.110:0/910280401 wait complete. 2024-09-15T03:41:36.121 INFO:teuthology.orchestra.run.smithi110.stderr:dumped all 2024-09-15T03:41:36.135 INFO:teuthology.orchestra.run.smithi110.stdout:{"pg_ready":true,"pg_map":{"version":166,"stamp":"2024-09-15T03:41:34.691448+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3570754347,"num_objects":882,"num_object_clones":0,"num_object_copies":1764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":882,"num_whiteouts":0,"num_read":2629,"num_read_kb":2634,"num_write":159122,"num_write_kb":5635367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161732,"ondisk_log_size":161732,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":9126520,"kb_used_data":6685304,"kb_used_omap":13,"kb_used_meta":2441138,"kb_avail":745848200,"statfs":{"total":773094113280,"available":763748556800,"internally_reserved":0,"allocated":6845751296,"data_stored":6840210502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2499726016},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003935"},"pg_stats":[{"pgid":"3.1f","version":"24'3049","reported_seq":3071,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:41:03.738227+0000","last_change":"2024-09-15T03:41:03.738193+0000","last_active":"2024-09-15T03:41:03.738227+0000","last_peered":"2024-09-15T03:41:03.738227+0000","last_clean":"2024-09-15T03:41:03.738227+0000","last_became_active":"2024-09-15T03:36:48.151625+0000","last_became_peered":"2024-09-15T03:36:48.151625+0000","last_unstale":"2024-09-15T03:41:03.738227+0000","last_undegraded":"2024-09-15T03:41:03.738227+0000","last_fullsized":"2024-09-15T03:41:03.738227+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'3049","last_scrub_stamp":"2024-09-15T03:41:03.738112+0000","last_deep_scrub":"24'3049","last_deep_scrub_stamp":"2024-09-15T03:41:03.738112+0000","last_clean_scrub_stamp":"2024-09-15T03:41:03.738112+0000","objects_scrubbed":24,"log_size":3049,"log_dups_size":0,"ondisk_log_size":3049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:06:44.541343+0000","scrub_duration":0.28826884200000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663296,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3019,"num_write_kb":136067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"24'6236","reported_seq":6564,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:56.787862+0000","last_change":"2024-09-15T03:40:56.787830+0000","last_active":"2024-09-15T03:40:56.787862+0000","last_peered":"2024-09-15T03:40:56.787862+0000","last_clean":"2024-09-15T03:40:56.787862+0000","last_became_active":"2024-09-15T03:36:48.149731+0000","last_became_peered":"2024-09-15T03:36:48.149731+0000","last_unstale":"2024-09-15T03:40:56.787862+0000","last_undegraded":"2024-09-15T03:40:56.787862+0000","last_fullsized":"2024-09-15T03:40:56.787862+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'6236","last_scrub_stamp":"2024-09-15T03:40:56.787721+0000","last_deep_scrub":"24'6236","last_deep_scrub_stamp":"2024-09-15T03:40:56.787721+0000","last_clean_scrub_stamp":"2024-09-15T03:40:56.787721+0000","objects_scrubbed":28,"log_size":6236,"log_dups_size":0,"ondisk_log_size":6236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:19:35.805960+0000","scrub_duration":0.37018249800000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113377280,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":1022,"num_read_kb":916,"num_write":6394,"num_write_kb":178051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"24'6319","reported_seq":6445,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:41:18.057705+0000","last_change":"2024-09-15T03:41:18.057661+0000","last_active":"2024-09-15T03:41:18.057705+0000","last_peered":"2024-09-15T03:41:18.057705+0000","last_clean":"2024-09-15T03:41:18.057705+0000","last_became_active":"2024-09-15T03:36:48.162085+0000","last_became_peered":"2024-09-15T03:36:48.162085+0000","last_unstale":"2024-09-15T03:41:18.057705+0000","last_undegraded":"2024-09-15T03:41:18.057705+0000","last_fullsized":"2024-09-15T03:41:18.057705+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'6319","last_scrub_stamp":"2024-09-15T03:41:18.057566+0000","last_deep_scrub":"24'6319","last_deep_scrub_stamp":"2024-09-15T03:41:18.057566+0000","last_clean_scrub_stamp":"2024-09-15T03:41:18.057566+0000","objects_scrubbed":31,"log_size":6319,"log_dups_size":0,"ondisk_log_size":6319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:24.326620+0000","scrub_duration":0.44227445799999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121898016,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":109,"num_read_kb":107,"num_write":6275,"num_write_kb":177699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"24'4678","reported_seq":4708,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:41:11.794374+0000","last_change":"2024-09-15T03:41:11.794328+0000","last_active":"2024-09-15T03:41:11.794374+0000","last_peered":"2024-09-15T03:41:11.794374+0000","last_clean":"2024-09-15T03:41:11.794374+0000","last_became_active":"2024-09-15T03:36:48.159272+0000","last_became_peered":"2024-09-15T03:36:48.159272+0000","last_unstale":"2024-09-15T03:41:11.794374+0000","last_undegraded":"2024-09-15T03:41:11.794374+0000","last_fullsized":"2024-09-15T03:41:11.794374+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'4678","last_scrub_stamp":"2024-09-15T03:41:11.794237+0000","last_deep_scrub":"24'4678","last_deep_scrub_stamp":"2024-09-15T03:41:11.794237+0000","last_clean_scrub_stamp":"2024-09-15T03:41:11.794237+0000","objects_scrubbed":22,"log_size":4678,"log_dups_size":0,"ondisk_log_size":4678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:52:12.546462+0000","scrub_duration":0.29342919200000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88866816,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":4665,"num_write_kb":169740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"24'4826","reported_seq":4849,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:56.758451+0000","last_change":"2024-09-15T03:40:56.758405+0000","last_active":"2024-09-15T03:40:56.758451+0000","last_peered":"2024-09-15T03:40:56.758451+0000","last_clean":"2024-09-15T03:40:56.758451+0000","last_became_active":"2024-09-15T03:36:48.150057+0000","last_became_peered":"2024-09-15T03:36:48.150057+0000","last_unstale":"2024-09-15T03:40:56.758451+0000","last_undegraded":"2024-09-15T03:40:56.758451+0000","last_fullsized":"2024-09-15T03:40:56.758451+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'4826","last_scrub_stamp":"2024-09-15T03:40:56.758284+0000","last_deep_scrub":"24'4826","last_deep_scrub_stamp":"2024-09-15T03:40:56.758284+0000","last_clean_scrub_stamp":"2024-09-15T03:40:56.758284+0000","objects_scrubbed":28,"log_size":4826,"log_dups_size":0,"ondisk_log_size":4826,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T12:47:57.646441+0000","scrub_duration":0.33452131800000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117440512,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4750,"num_write_kb":185221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"24'4732","reported_seq":4759,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:55.144629+0000","last_change":"2024-09-15T03:40:55.144557+0000","last_active":"2024-09-15T03:40:55.144629+0000","last_peered":"2024-09-15T03:40:55.144629+0000","last_clean":"2024-09-15T03:40:55.144629+0000","last_became_active":"2024-09-15T03:36:48.144097+0000","last_became_peered":"2024-09-15T03:36:48.144097+0000","last_unstale":"2024-09-15T03:40:55.144629+0000","last_undegraded":"2024-09-15T03:40:55.144629+0000","last_fullsized":"2024-09-15T03:40:55.144629+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'4732","last_scrub_stamp":"2024-09-15T03:40:55.144302+0000","last_deep_scrub":"24'4732","last_deep_scrub_stamp":"2024-09-15T03:40:55.144302+0000","last_clean_scrub_stamp":"2024-09-15T03:40:55.144302+0000","objects_scrubbed":18,"log_size":4732,"log_dups_size":0,"ondisk_log_size":4732,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:12:19.069563+0000","scrub_duration":0.215515611,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68681728,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":22,"num_read_kb":16,"num_write":4699,"num_write_kb":143618,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"24'4521","reported_seq":4545,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:41:07.568108+0000","last_change":"2024-09-15T03:41:07.568045+0000","last_active":"2024-09-15T03:41:07.568108+0000","last_peered":"2024-09-15T03:41:07.568108+0000","last_clean":"2024-09-15T03:41:07.568108+0000","last_became_active":"2024-09-15T03:36:48.142085+0000","last_became_peered":"2024-09-15T03:36:48.142085+0000","last_unstale":"2024-09-15T03:41:07.568108+0000","last_undegraded":"2024-09-15T03:41:07.568108+0000","last_fullsized":"2024-09-15T03:41:07.568108+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'4521","last_scrub_stamp":"2024-09-15T03:41:07.567893+0000","last_deep_scrub":"24'4521","last_deep_scrub_stamp":"2024-09-15T03:41:07.567893+0000","last_clean_scrub_stamp":"2024-09-15T03:41:07.567893+0000","objects_scrubbed":28,"log_size":4521,"log_dups_size":0,"ondisk_log_size":4521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:13:47.082289+0000","scrub_duration":0.23014422900000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":111872344,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":4448,"num_write_kb":183583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"24'7233","reported_seq":7260,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:41:03.795098+0000","last_change":"2024-09-15T03:41:03.795039+0000","last_active":"2024-09-15T03:41:03.795098+0000","last_peered":"2024-09-15T03:41:03.795098+0000","last_clean":"2024-09-15T03:41:03.795098+0000","last_became_active":"2024-09-15T03:36:48.138049+0000","last_became_peered":"2024-09-15T03:36:48.138049+0000","last_unstale":"2024-09-15T03:41:03.795098+0000","last_undegraded":"2024-09-15T03:41:03.795098+0000","last_fullsized":"2024-09-15T03:41:03.795098+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'7233","last_scrub_stamp":"2024-09-15T03:41:03.794874+0000","last_deep_scrub":"24'7233","last_deep_scrub_stamp":"2024-09-15T03:41:03.794874+0000","last_clean_scrub_stamp":"2024-09-15T03:41:03.794874+0000","objects_scrubbed":28,"log_size":7233,"log_dups_size":0,"ondisk_log_size":7233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T13:50:20.679143+0000","scrub_duration":0.27943716800000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113246720,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":15,"num_read_kb":11,"num_write":7218,"num_write_kb":193736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"24'7897","reported_seq":7927,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:55.869062+0000","last_change":"2024-09-15T03:40:55.869035+0000","last_active":"2024-09-15T03:40:55.869062+0000","last_peered":"2024-09-15T03:40:55.869062+0000","last_clean":"2024-09-15T03:40:55.869062+0000","last_became_active":"2024-09-15T03:36:48.149792+0000","last_became_peered":"2024-09-15T03:36:48.149792+0000","last_unstale":"2024-09-15T03:40:55.869062+0000","last_undegraded":"2024-09-15T03:40:55.869062+0000","last_fullsized":"2024-09-15T03:40:55.869062+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'7897","last_scrub_stamp":"2024-09-15T03:40:55.868958+0000","last_deep_scrub":"24'7897","last_deep_scrub_stamp":"2024-09-15T03:40:55.868958+0000","last_clean_scrub_stamp":"2024-09-15T03:40:55.868958+0000","objects_scrubbed":35,"log_size":7897,"log_dups_size":0,"ondisk_log_size":7897,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T13:15:33.314556+0000","scrub_duration":0.44187000399999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":146800640,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":7874,"num_write_kb":194640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"24'2428","reported_seq":2453,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:41:03.743187+0000","last_change":"2024-09-15T03:41:03.743157+0000","last_active":"2024-09-15T03:41:03.743187+0000","last_peered":"2024-09-15T03:41:03.743187+0000","last_clean":"2024-09-15T03:41:03.743187+0000","last_became_active":"2024-09-15T03:36:48.154800+0000","last_became_peered":"2024-09-15T03:36:48.154800+0000","last_unstale":"2024-09-15T03:41:03.743187+0000","last_undegraded":"2024-09-15T03:41:03.743187+0000","last_fullsized":"2024-09-15T03:41:03.743187+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'2428","last_scrub_stamp":"2024-09-15T03:41:03.743030+0000","last_deep_scrub":"24'2428","last_deep_scrub_stamp":"2024-09-15T03:41:03.743030+0000","last_clean_scrub_stamp":"2024-09-15T03:41:03.743030+0000","objects_scrubbed":19,"log_size":2428,"log_dups_size":0,"ondisk_log_size":2428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:57:26.522200+0000","scrub_duration":0.22485928499999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":2388,"num_write_kb":127362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"24'6333","reported_seq":6362,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:41:21.683914+0000","last_change":"2024-09-15T03:41:21.683881+0000","last_active":"2024-09-15T03:41:21.683914+0000","last_peered":"2024-09-15T03:41:21.683914+0000","last_clean":"2024-09-15T03:41:21.683914+0000","last_became_active":"2024-09-15T03:36:48.152191+0000","last_became_peered":"2024-09-15T03:36:48.152191+0000","last_unstale":"2024-09-15T03:41:21.683914+0000","last_undegraded":"2024-09-15T03:41:21.683914+0000","last_fullsized":"2024-09-15T03:41:21.683914+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'6333","last_scrub_stamp":"2024-09-15T03:41:21.683792+0000","last_deep_scrub":"24'6333","last_deep_scrub_stamp":"2024-09-15T03:41:21.683792+0000","last_clean_scrub_stamp":"2024-09-15T03:41:21.683792+0000","objects_scrubbed":32,"log_size":6333,"log_dups_size":0,"ondisk_log_size":6333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:21:55.222256+0000","scrub_duration":0.354801639,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":124518928,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":19,"num_read_kb":14,"num_write":6316,"num_write_kb":168780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"24'5415","reported_seq":5441,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:56.272066+0000","last_change":"2024-09-15T03:40:56.272033+0000","last_active":"2024-09-15T03:40:56.272066+0000","last_peered":"2024-09-15T03:40:56.272066+0000","last_clean":"2024-09-15T03:40:56.272066+0000","last_became_active":"2024-09-15T03:36:48.153926+0000","last_became_peered":"2024-09-15T03:36:48.153926+0000","last_unstale":"2024-09-15T03:40:56.272066+0000","last_undegraded":"2024-09-15T03:40:56.272066+0000","last_fullsized":"2024-09-15T03:40:56.272066+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'5415","last_scrub_stamp":"2024-09-15T03:40:56.271945+0000","last_deep_scrub":"24'5415","last_deep_scrub_stamp":"2024-09-15T03:40:56.271945+0000","last_clean_scrub_stamp":"2024-09-15T03:40:56.271945+0000","objects_scrubbed":28,"log_size":5415,"log_dups_size":0,"ondisk_log_size":5415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:00:32.277556+0000","scrub_duration":0.37861801299999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109121016,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":5342,"num_write_kb":144586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":31,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:58.377582+0000","last_change":"2024-09-15T03:40:58.377503+0000","last_active":"2024-09-15T03:40:58.377582+0000","last_peered":"2024-09-15T03:40:58.377582+0000","last_clean":"2024-09-15T03:40:58.377582+0000","last_became_active":"2024-09-15T03:36:40.672828+0000","last_became_peered":"2024-09-15T03:36:40.672828+0000","last_unstale":"2024-09-15T03:40:58.377582+0000","last_undegraded":"2024-09-15T03:40:58.377582+0000","last_fullsized":"2024-09-15T03:40:58.377582+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:40:58.377270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:40:58.377270+0000","last_clean_scrub_stamp":"2024-09-15T03:40:58.377270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:22:36.982400+0000","scrub_duration":0.003092628,"objects_trimmed":0,"snaptrim_duration":0.00064069099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"24'6015","reported_seq":6061,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:53.674105+0000","last_change":"2024-09-15T03:40:53.674065+0000","last_active":"2024-09-15T03:40:53.674105+0000","last_peered":"2024-09-15T03:40:53.674105+0000","last_clean":"2024-09-15T03:40:53.674105+0000","last_became_active":"2024-09-15T03:36:48.143177+0000","last_became_peered":"2024-09-15T03:36:48.143177+0000","last_unstale":"2024-09-15T03:40:53.674105+0000","last_undegraded":"2024-09-15T03:40:53.674105+0000","last_fullsized":"2024-09-15T03:40:53.674105+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'6015","last_scrub_stamp":"2024-09-15T03:40:53.673943+0000","last_deep_scrub":"24'6015","last_deep_scrub_stamp":"2024-09-15T03:40:53.673943+0000","last_clean_scrub_stamp":"2024-09-15T03:40:53.673943+0000","objects_scrubbed":33,"log_size":6015,"log_dups_size":0,"ondisk_log_size":6015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T13:33:53.399179+0000","scrub_duration":0.274737495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":134639616,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":92,"num_read_kb":80,"num_write":5989,"num_write_kb":192351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":31,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:53.212546+0000","last_change":"2024-09-15T03:40:53.212457+0000","last_active":"2024-09-15T03:40:53.212546+0000","last_peered":"2024-09-15T03:40:53.212546+0000","last_clean":"2024-09-15T03:40:53.212546+0000","last_became_active":"2024-09-15T03:36:40.671677+0000","last_became_peered":"2024-09-15T03:36:40.671677+0000","last_unstale":"2024-09-15T03:40:53.212546+0000","last_undegraded":"2024-09-15T03:40:53.212546+0000","last_fullsized":"2024-09-15T03:40:53.212546+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:40:53.212248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:40:53.212248+0000","last_clean_scrub_stamp":"2024-09-15T03:40:53.212248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:33:29.916166+0000","scrub_duration":0.003136242,"objects_trimmed":0,"snaptrim_duration":0.000861089,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"24'5430","reported_seq":5453,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:58.788643+0000","last_change":"2024-09-15T03:40:58.788611+0000","last_active":"2024-09-15T03:40:58.788643+0000","last_peered":"2024-09-15T03:40:58.788643+0000","last_clean":"2024-09-15T03:40:58.788643+0000","last_became_active":"2024-09-15T03:36:48.151844+0000","last_became_peered":"2024-09-15T03:36:48.151844+0000","last_unstale":"2024-09-15T03:40:58.788643+0000","last_undegraded":"2024-09-15T03:40:58.788643+0000","last_fullsized":"2024-09-15T03:40:58.788643+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'5430","last_scrub_stamp":"2024-09-15T03:40:58.788521+0000","last_deep_scrub":"24'5430","last_deep_scrub_stamp":"2024-09-15T03:40:58.788521+0000","last_clean_scrub_stamp":"2024-09-15T03:40:58.788521+0000","objects_scrubbed":28,"log_size":5430,"log_dups_size":0,"ondisk_log_size":5430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:42:39.962811+0000","scrub_duration":0.34940842100000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":115998720,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5352,"num_write_kb":174984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":31,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:41:09.338334+0000","last_change":"2024-09-15T03:41:09.338237+0000","last_active":"2024-09-15T03:41:09.338334+0000","last_peered":"2024-09-15T03:41:09.338334+0000","last_clean":"2024-09-15T03:41:09.338334+0000","last_became_active":"2024-09-15T03:36:41.079216+0000","last_became_peered":"2024-09-15T03:36:41.079216+0000","last_unstale":"2024-09-15T03:41:09.338334+0000","last_undegraded":"2024-09-15T03:41:09.338334+0000","last_fullsized":"2024-09-15T03:41:09.338334+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:41:09.338037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:41:09.338037+0000","last_clean_scrub_stamp":"2024-09-15T03:41:09.338037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T13:23:59.168130+0000","scrub_duration":0.0036244739999999999,"objects_trimmed":0,"snaptrim_duration":0.00062568200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"24'7307","reported_seq":7341,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:41:06.686821+0000","last_change":"2024-09-15T03:41:06.686777+0000","last_active":"2024-09-15T03:41:06.686821+0000","last_peered":"2024-09-15T03:41:06.686821+0000","last_clean":"2024-09-15T03:41:06.686821+0000","last_became_active":"2024-09-15T03:36:48.216337+0000","last_became_peered":"2024-09-15T03:36:48.216337+0000","last_unstale":"2024-09-15T03:41:06.686821+0000","last_undegraded":"2024-09-15T03:41:06.686821+0000","last_fullsized":"2024-09-15T03:41:06.686821+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'7307","last_scrub_stamp":"2024-09-15T03:41:06.686644+0000","last_deep_scrub":"24'7307","last_deep_scrub_stamp":"2024-09-15T03:41:06.686644+0000","last_clean_scrub_stamp":"2024-09-15T03:41:06.686644+0000","objects_scrubbed":37,"log_size":7307,"log_dups_size":0,"ondisk_log_size":7307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:19:17.191909+0000","scrub_duration":0.35128014299999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":146931712,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":45,"num_read_kb":34,"num_write":7284,"num_write_kb":246402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":31,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:58.302534+0000","last_change":"2024-09-15T03:40:58.302464+0000","last_active":"2024-09-15T03:40:58.302534+0000","last_peered":"2024-09-15T03:40:58.302534+0000","last_clean":"2024-09-15T03:40:58.302534+0000","last_became_active":"2024-09-15T03:36:41.077266+0000","last_became_peered":"2024-09-15T03:36:41.077266+0000","last_unstale":"2024-09-15T03:40:58.302534+0000","last_undegraded":"2024-09-15T03:40:58.302534+0000","last_fullsized":"2024-09-15T03:40:58.302534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:40:58.302274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:40:58.302274+0000","last_clean_scrub_stamp":"2024-09-15T03:40:58.302274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:00:53.493431+0000","scrub_duration":0.0031430619999999999,"objects_trimmed":0,"snaptrim_duration":0.00067514400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"24'3304","reported_seq":3326,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:41:00.727165+0000","last_change":"2024-09-15T03:41:00.727138+0000","last_active":"2024-09-15T03:41:00.727165+0000","last_peered":"2024-09-15T03:41:00.727165+0000","last_clean":"2024-09-15T03:41:00.727165+0000","last_became_active":"2024-09-15T03:36:48.151520+0000","last_became_peered":"2024-09-15T03:36:48.151520+0000","last_unstale":"2024-09-15T03:41:00.727165+0000","last_undegraded":"2024-09-15T03:41:00.727165+0000","last_fullsized":"2024-09-15T03:41:00.727165+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'3304","last_scrub_stamp":"2024-09-15T03:41:00.727061+0000","last_deep_scrub":"24'3304","last_deep_scrub_stamp":"2024-09-15T03:41:00.727061+0000","last_clean_scrub_stamp":"2024-09-15T03:41:00.727061+0000","objects_scrubbed":22,"log_size":3304,"log_dups_size":0,"ondisk_log_size":3304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T15:01:42.696558+0000","scrub_duration":0.25260938199999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3278,"num_write_kb":144390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"24'3476","reported_seq":3513,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:58.195729+0000","last_change":"2024-09-15T03:40:58.195683+0000","last_active":"2024-09-15T03:40:58.195729+0000","last_peered":"2024-09-15T03:40:58.195729+0000","last_clean":"2024-09-15T03:40:58.195729+0000","last_became_active":"2024-09-15T03:36:48.145845+0000","last_became_peered":"2024-09-15T03:36:48.145845+0000","last_unstale":"2024-09-15T03:40:58.195729+0000","last_undegraded":"2024-09-15T03:40:58.195729+0000","last_fullsized":"2024-09-15T03:40:58.195729+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'3476","last_scrub_stamp":"2024-09-15T03:40:58.195566+0000","last_deep_scrub":"24'3476","last_deep_scrub_stamp":"2024-09-15T03:40:58.195566+0000","last_clean_scrub_stamp":"2024-09-15T03:40:58.195566+0000","objects_scrubbed":27,"log_size":3476,"log_dups_size":0,"ondisk_log_size":3476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:57:26.163454+0000","scrub_duration":0.311787747,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113246208,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":50,"num_read_kb":41,"num_write":3479,"num_write_kb":144783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"24'3","reported_seq":34,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:57.489841+0000","last_change":"2024-09-15T03:40:57.489767+0000","last_active":"2024-09-15T03:40:57.489841+0000","last_peered":"2024-09-15T03:40:57.489841+0000","last_clean":"2024-09-15T03:40:57.489841+0000","last_became_active":"2024-09-15T03:36:40.673743+0000","last_became_peered":"2024-09-15T03:36:40.673743+0000","last_unstale":"2024-09-15T03:40:57.489841+0000","last_undegraded":"2024-09-15T03:40:57.489841+0000","last_fullsized":"2024-09-15T03:40:57.489841+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'3","last_scrub_stamp":"2024-09-15T03:40:57.489545+0000","last_deep_scrub":"24'3","last_deep_scrub_stamp":"2024-09-15T03:40:57.489545+0000","last_clean_scrub_stamp":"2024-09-15T03:40:57.489545+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:18:35.777966+0000","scrub_duration":0.006897373,"objects_trimmed":0,"snaptrim_duration":0.002159518,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"24'4749","reported_seq":4786,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:41:00.645265+0000","last_change":"2024-09-15T03:41:00.645216+0000","last_active":"2024-09-15T03:41:00.645265+0000","last_peered":"2024-09-15T03:41:00.645265+0000","last_clean":"2024-09-15T03:41:00.645265+0000","last_became_active":"2024-09-15T03:36:48.216269+0000","last_became_peered":"2024-09-15T03:36:48.216269+0000","last_unstale":"2024-09-15T03:41:00.645265+0000","last_undegraded":"2024-09-15T03:41:00.645265+0000","last_fullsized":"2024-09-15T03:41:00.645265+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'4749","last_scrub_stamp":"2024-09-15T03:41:00.645075+0000","last_deep_scrub":"24'4749","last_deep_scrub_stamp":"2024-09-15T03:41:00.645075+0000","last_clean_scrub_stamp":"2024-09-15T03:41:00.645075+0000","objects_scrubbed":37,"log_size":4749,"log_dups_size":0,"ondisk_log_size":4749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T15:04:15.528288+0000","scrub_duration":0.30872137900000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":142877200,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":51,"num_read_kb":164,"num_write":4698,"num_write_kb":182422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":31,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:53.651919+0000","last_change":"2024-09-15T03:40:53.651875+0000","last_active":"2024-09-15T03:40:53.651919+0000","last_peered":"2024-09-15T03:40:53.651919+0000","last_clean":"2024-09-15T03:40:53.651919+0000","last_became_active":"2024-09-15T03:36:40.669392+0000","last_became_peered":"2024-09-15T03:36:40.669392+0000","last_unstale":"2024-09-15T03:40:53.651919+0000","last_undegraded":"2024-09-15T03:40:53.651919+0000","last_fullsized":"2024-09-15T03:40:53.651919+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:40:53.651715+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:40:53.651715+0000","last_clean_scrub_stamp":"2024-09-15T03:40:53.651715+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:08:35.720046+0000","scrub_duration":0.0021291890000000001,"objects_trimmed":0,"snaptrim_duration":0.00059629799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"24'3251","reported_seq":3281,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:54.648696+0000","last_change":"2024-09-15T03:40:54.648656+0000","last_active":"2024-09-15T03:40:54.648696+0000","last_peered":"2024-09-15T03:40:54.648696+0000","last_clean":"2024-09-15T03:40:54.648696+0000","last_became_active":"2024-09-15T03:36:48.144313+0000","last_became_peered":"2024-09-15T03:36:48.144313+0000","last_unstale":"2024-09-15T03:40:54.648696+0000","last_undegraded":"2024-09-15T03:40:54.648696+0000","last_fullsized":"2024-09-15T03:40:54.648696+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'3251","last_scrub_stamp":"2024-09-15T03:40:54.648582+0000","last_deep_scrub":"24'3251","last_deep_scrub_stamp":"2024-09-15T03:40:54.648582+0000","last_clean_scrub_stamp":"2024-09-15T03:40:54.648582+0000","objects_scrubbed":23,"log_size":3251,"log_dups_size":0,"ondisk_log_size":3251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:24:02.179680+0000","scrub_duration":0.24652201300000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92405760,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":33,"num_read_kb":87,"num_write":3327,"num_write_kb":128260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":31,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:41:07.352881+0000","last_change":"2024-09-15T03:41:07.352809+0000","last_active":"2024-09-15T03:41:07.352881+0000","last_peered":"2024-09-15T03:41:07.352881+0000","last_clean":"2024-09-15T03:41:07.352881+0000","last_became_active":"2024-09-15T03:36:40.672416+0000","last_became_peered":"2024-09-15T03:36:40.672416+0000","last_unstale":"2024-09-15T03:41:07.352881+0000","last_undegraded":"2024-09-15T03:41:07.352881+0000","last_fullsized":"2024-09-15T03:41:07.352881+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:41:07.352573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:41:07.352573+0000","last_clean_scrub_stamp":"2024-09-15T03:41:07.352573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:17:17.658369+0000","scrub_duration":0.0038595510000000001,"objects_trimmed":0,"snaptrim_duration":0.00065187600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"24'3123","reported_seq":3145,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:53.585199+0000","last_change":"2024-09-15T03:40:53.585158+0000","last_active":"2024-09-15T03:40:53.585199+0000","last_peered":"2024-09-15T03:40:53.585199+0000","last_clean":"2024-09-15T03:40:53.585199+0000","last_became_active":"2024-09-15T03:36:48.150693+0000","last_became_peered":"2024-09-15T03:36:48.150693+0000","last_unstale":"2024-09-15T03:40:53.585199+0000","last_undegraded":"2024-09-15T03:40:53.585199+0000","last_fullsized":"2024-09-15T03:40:53.585199+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'3123","last_scrub_stamp":"2024-09-15T03:40:53.585070+0000","last_deep_scrub":"24'3123","last_deep_scrub_stamp":"2024-09-15T03:40:53.585070+0000","last_clean_scrub_stamp":"2024-09-15T03:40:53.585070+0000","objects_scrubbed":16,"log_size":3123,"log_dups_size":0,"ondisk_log_size":3123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:13:52.170594+0000","scrub_duration":0.174875789,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67108864,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3105,"num_write_kb":103402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"24'2","reported_seq":33,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:58.467469+0000","last_change":"2024-09-15T03:40:58.467398+0000","last_active":"2024-09-15T03:40:58.467469+0000","last_peered":"2024-09-15T03:40:58.467469+0000","last_clean":"2024-09-15T03:40:58.467469+0000","last_became_active":"2024-09-15T03:36:40.674442+0000","last_became_peered":"2024-09-15T03:36:40.674442+0000","last_unstale":"2024-09-15T03:40:58.467469+0000","last_undegraded":"2024-09-15T03:40:58.467469+0000","last_fullsized":"2024-09-15T03:40:58.467469+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'2","last_scrub_stamp":"2024-09-15T03:40:58.467214+0000","last_deep_scrub":"24'2","last_deep_scrub_stamp":"2024-09-15T03:40:58.467214+0000","last_clean_scrub_stamp":"2024-09-15T03:40:58.467214+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:42:10.462331+0000","scrub_duration":0.004548524,"objects_trimmed":0,"snaptrim_duration":0.00067001500000000004,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"24'74","reported_seq":120,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:59.495959+0000","last_change":"2024-09-15T03:40:59.495892+0000","last_active":"2024-09-15T03:40:59.495959+0000","last_peered":"2024-09-15T03:40:59.495959+0000","last_clean":"2024-09-15T03:40:59.495959+0000","last_became_active":"2024-09-15T03:36:38.813490+0000","last_became_peered":"2024-09-15T03:36:38.813490+0000","last_unstale":"2024-09-15T03:40:59.495959+0000","last_undegraded":"2024-09-15T03:40:59.495959+0000","last_fullsized":"2024-09-15T03:40:59.495959+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'74","last_scrub_stamp":"2024-09-15T03:40:59.495711+0000","last_deep_scrub":"24'74","last_deep_scrub_stamp":"2024-09-15T03:40:59.495711+0000","last_clean_scrub_stamp":"2024-09-15T03:40:59.495711+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:54:47.135962+0000","scrub_duration":0.0079286540000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"24'5218","reported_seq":5245,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:59.654581+0000","last_change":"2024-09-15T03:40:59.654544+0000","last_active":"2024-09-15T03:40:59.654581+0000","last_peered":"2024-09-15T03:40:59.654581+0000","last_clean":"2024-09-15T03:40:59.654581+0000","last_became_active":"2024-09-15T03:36:48.142313+0000","last_became_peered":"2024-09-15T03:36:48.142313+0000","last_unstale":"2024-09-15T03:40:59.654581+0000","last_undegraded":"2024-09-15T03:40:59.654581+0000","last_fullsized":"2024-09-15T03:40:59.654581+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'5218","last_scrub_stamp":"2024-09-15T03:40:59.654440+0000","last_deep_scrub":"24'5218","last_deep_scrub_stamp":"2024-09-15T03:40:59.654440+0000","last_clean_scrub_stamp":"2024-09-15T03:40:59.654440+0000","objects_scrubbed":28,"log_size":5218,"log_dups_size":0,"ondisk_log_size":5218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T12:51:50.746216+0000","scrub_duration":0.26478074600000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":116760576,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":12,"num_read_kb":135,"num_write":5130,"num_write_kb":177200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"24'7969","reported_seq":8107,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:54.221970+0000","last_change":"2024-09-15T03:40:54.221919+0000","last_active":"2024-09-15T03:40:54.221970+0000","last_peered":"2024-09-15T03:40:54.221970+0000","last_clean":"2024-09-15T03:40:54.221970+0000","last_became_active":"2024-09-15T03:36:48.218083+0000","last_became_peered":"2024-09-15T03:36:48.218083+0000","last_unstale":"2024-09-15T03:40:54.221970+0000","last_undegraded":"2024-09-15T03:40:54.221970+0000","last_fullsized":"2024-09-15T03:40:54.221970+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'7969","last_scrub_stamp":"2024-09-15T03:40:54.221766+0000","last_deep_scrub":"24'7969","last_deep_scrub_stamp":"2024-09-15T03:40:54.221766+0000","last_clean_scrub_stamp":"2024-09-15T03:40:54.221766+0000","objects_scrubbed":25,"log_size":7969,"log_dups_size":0,"ondisk_log_size":7969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:57:37.082720+0000","scrub_duration":0.32031514999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663808,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":381,"num_read_kb":409,"num_write":5985,"num_write_kb":152253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"24'5871","reported_seq":5914,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:41:06.714875+0000","last_change":"2024-09-15T03:41:06.714838+0000","last_active":"2024-09-15T03:41:06.714875+0000","last_peered":"2024-09-15T03:41:06.714875+0000","last_clean":"2024-09-15T03:41:06.714875+0000","last_became_active":"2024-09-15T03:36:48.155263+0000","last_became_peered":"2024-09-15T03:36:48.155263+0000","last_unstale":"2024-09-15T03:41:06.714875+0000","last_undegraded":"2024-09-15T03:41:06.714875+0000","last_fullsized":"2024-09-15T03:41:06.714875+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'5871","last_scrub_stamp":"2024-09-15T03:41:06.714714+0000","last_deep_scrub":"24'5871","last_deep_scrub_stamp":"2024-09-15T03:41:06.714714+0000","last_clean_scrub_stamp":"2024-09-15T03:41:06.714714+0000","objects_scrubbed":34,"log_size":5871,"log_dups_size":0,"ondisk_log_size":5871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:14:56.633769+0000","scrub_duration":0.46467073599999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":137103712,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":68,"num_read_kb":57,"num_write":5841,"num_write_kb":205281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"24'3170","reported_seq":3192,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:41:02.654633+0000","last_change":"2024-09-15T03:41:02.654604+0000","last_active":"2024-09-15T03:41:02.654633+0000","last_peered":"2024-09-15T03:41:02.654633+0000","last_clean":"2024-09-15T03:41:02.654633+0000","last_became_active":"2024-09-15T03:36:48.137589+0000","last_became_peered":"2024-09-15T03:36:48.137589+0000","last_unstale":"2024-09-15T03:41:02.654633+0000","last_undegraded":"2024-09-15T03:41:02.654633+0000","last_fullsized":"2024-09-15T03:41:02.654633+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'3170","last_scrub_stamp":"2024-09-15T03:41:02.654523+0000","last_deep_scrub":"24'3170","last_deep_scrub_stamp":"2024-09-15T03:41:02.654523+0000","last_clean_scrub_stamp":"2024-09-15T03:41:02.654523+0000","objects_scrubbed":20,"log_size":3170,"log_dups_size":0,"ondisk_log_size":3170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T13:52:24.275895+0000","scrub_duration":0.156042232,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3149,"num_write_kb":152324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"24'4762","reported_seq":4802,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:41:04.789904+0000","last_change":"2024-09-15T03:41:04.789872+0000","last_active":"2024-09-15T03:41:04.789904+0000","last_peered":"2024-09-15T03:41:04.789904+0000","last_clean":"2024-09-15T03:41:04.789904+0000","last_became_active":"2024-09-15T03:36:48.151365+0000","last_became_peered":"2024-09-15T03:36:48.151365+0000","last_unstale":"2024-09-15T03:41:04.789904+0000","last_undegraded":"2024-09-15T03:41:04.789904+0000","last_fullsized":"2024-09-15T03:41:04.789904+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'4762","last_scrub_stamp":"2024-09-15T03:41:04.789781+0000","last_deep_scrub":"24'4762","last_deep_scrub_stamp":"2024-09-15T03:41:04.789781+0000","last_clean_scrub_stamp":"2024-09-15T03:41:04.789781+0000","objects_scrubbed":30,"log_size":4762,"log_dups_size":0,"ondisk_log_size":4762,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T12:58:01.226468+0000","scrub_duration":0.32063545500000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121635328,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":72,"num_read_kb":60,"num_write":4709,"num_write_kb":215880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"24'5546","reported_seq":5573,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:41:18.744315+0000","last_change":"2024-09-15T03:41:18.744284+0000","last_active":"2024-09-15T03:41:18.744315+0000","last_peered":"2024-09-15T03:41:18.744315+0000","last_clean":"2024-09-15T03:41:18.744315+0000","last_became_active":"2024-09-15T03:36:48.154955+0000","last_became_peered":"2024-09-15T03:36:48.154955+0000","last_unstale":"2024-09-15T03:41:18.744315+0000","last_undegraded":"2024-09-15T03:41:18.744315+0000","last_fullsized":"2024-09-15T03:41:18.744315+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'5546","last_scrub_stamp":"2024-09-15T03:41:18.744200+0000","last_deep_scrub":"24'5546","last_deep_scrub_stamp":"2024-09-15T03:41:18.744200+0000","last_clean_scrub_stamp":"2024-09-15T03:41:18.744200+0000","objects_scrubbed":28,"log_size":5546,"log_dups_size":0,"ondisk_log_size":5546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:29:31.464085+0000","scrub_duration":0.37914669600000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117440512,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":5499,"num_write_kb":248958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"24'4115","reported_seq":4232,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:41:17.701670+0000","last_change":"2024-09-15T03:41:17.701621+0000","last_active":"2024-09-15T03:41:17.701670+0000","last_peered":"2024-09-15T03:41:17.701670+0000","last_clean":"2024-09-15T03:41:17.701670+0000","last_became_active":"2024-09-15T03:36:48.162401+0000","last_became_peered":"2024-09-15T03:36:48.162401+0000","last_unstale":"2024-09-15T03:41:17.701670+0000","last_undegraded":"2024-09-15T03:41:17.701670+0000","last_fullsized":"2024-09-15T03:41:17.701670+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'4115","last_scrub_stamp":"2024-09-15T03:41:17.701512+0000","last_deep_scrub":"24'4115","last_deep_scrub_stamp":"2024-09-15T03:41:17.701512+0000","last_clean_scrub_stamp":"2024-09-15T03:41:17.701512+0000","objects_scrubbed":25,"log_size":4115,"log_dups_size":0,"ondisk_log_size":4115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:51:41.411597+0000","scrub_duration":0.31205429899999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104857600,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":310,"num_read_kb":242,"num_write":4350,"num_write_kb":177773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"24'3647","reported_seq":3689,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:41:16.801802+0000","last_change":"2024-09-15T03:41:16.801719+0000","last_active":"2024-09-15T03:41:16.801802+0000","last_peered":"2024-09-15T03:41:16.801802+0000","last_clean":"2024-09-15T03:41:16.801802+0000","last_became_active":"2024-09-15T03:36:48.161897+0000","last_became_peered":"2024-09-15T03:36:48.161897+0000","last_unstale":"2024-09-15T03:41:16.801802+0000","last_undegraded":"2024-09-15T03:41:16.801802+0000","last_fullsized":"2024-09-15T03:41:16.801802+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'3647","last_scrub_stamp":"2024-09-15T03:41:16.801517+0000","last_deep_scrub":"24'3647","last_deep_scrub_stamp":"2024-09-15T03:41:16.801517+0000","last_clean_scrub_stamp":"2024-09-15T03:41:16.801517+0000","objects_scrubbed":31,"log_size":3647,"log_dups_size":0,"ondisk_log_size":3647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:23:46.647967+0000","scrub_duration":0.369117431,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":130023424,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":75,"num_read_kb":60,"num_write":3605,"num_write_kb":185220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"24'4082","reported_seq":4105,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:41:10.535847+0000","last_change":"2024-09-15T03:41:10.535781+0000","last_active":"2024-09-15T03:41:10.535847+0000","last_peered":"2024-09-15T03:41:10.535847+0000","last_clean":"2024-09-15T03:41:10.535847+0000","last_became_active":"2024-09-15T03:36:48.149651+0000","last_became_peered":"2024-09-15T03:36:48.149651+0000","last_unstale":"2024-09-15T03:41:10.535847+0000","last_undegraded":"2024-09-15T03:41:10.535847+0000","last_fullsized":"2024-09-15T03:41:10.535847+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'4082","last_scrub_stamp":"2024-09-15T03:41:10.535652+0000","last_deep_scrub":"24'4082","last_deep_scrub_stamp":"2024-09-15T03:41:10.535652+0000","last_clean_scrub_stamp":"2024-09-15T03:41:10.535652+0000","objects_scrubbed":27,"log_size":4082,"log_dups_size":0,"ondisk_log_size":4082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:57:03.680831+0000","scrub_duration":0.33710412099999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":111869952,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4006,"num_write_kb":198851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"24'2817","reported_seq":2855,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:41:10.604994+0000","last_change":"2024-09-15T03:41:10.604946+0000","last_active":"2024-09-15T03:41:10.604994+0000","last_peered":"2024-09-15T03:41:10.604994+0000","last_clean":"2024-09-15T03:41:10.604994+0000","last_became_active":"2024-09-15T03:36:48.159380+0000","last_became_peered":"2024-09-15T03:36:48.159380+0000","last_unstale":"2024-09-15T03:41:10.604994+0000","last_undegraded":"2024-09-15T03:41:10.604994+0000","last_fullsized":"2024-09-15T03:41:10.604994+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'2817","last_scrub_stamp":"2024-09-15T03:41:10.604847+0000","last_deep_scrub":"24'2817","last_deep_scrub_stamp":"2024-09-15T03:41:10.604847+0000","last_clean_scrub_stamp":"2024-09-15T03:41:10.604847+0000","objects_scrubbed":20,"log_size":2817,"log_dups_size":0,"ondisk_log_size":2817,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:25:40.151554+0000","scrub_duration":0.28442361100000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":50,"num_read_kb":41,"num_write":2819,"num_write_kb":144138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"24'5748","reported_seq":5776,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:40:52.818187+0000","last_change":"2024-09-15T03:40:52.818147+0000","last_active":"2024-09-15T03:40:52.818187+0000","last_peered":"2024-09-15T03:40:52.818187+0000","last_clean":"2024-09-15T03:40:52.818187+0000","last_became_active":"2024-09-15T03:36:48.150784+0000","last_became_peered":"2024-09-15T03:36:48.150784+0000","last_unstale":"2024-09-15T03:40:52.818187+0000","last_undegraded":"2024-09-15T03:40:52.818187+0000","last_fullsized":"2024-09-15T03:40:52.818187+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'5748","last_scrub_stamp":"2024-09-15T03:40:52.818031+0000","last_deep_scrub":"24'5748","last_deep_scrub_stamp":"2024-09-15T03:40:52.818031+0000","last_clean_scrub_stamp":"2024-09-15T03:40:52.818031+0000","objects_scrubbed":34,"log_size":5748,"log_dups_size":0,"ondisk_log_size":5748,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:21.091184+0000","scrub_duration":0.37681035499999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":138543104,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":22,"num_read_kb":16,"num_write":5677,"num_write_kb":255624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"24'8386","reported_seq":8418,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T03:41:13.830529+0000","last_change":"2024-09-15T03:41:13.830473+0000","last_active":"2024-09-15T03:41:13.830529+0000","last_peered":"2024-09-15T03:41:13.830529+0000","last_clean":"2024-09-15T03:41:13.830529+0000","last_became_active":"2024-09-15T03:36:48.159470+0000","last_became_peered":"2024-09-15T03:36:48.159470+0000","last_unstale":"2024-09-15T03:41:13.830529+0000","last_undegraded":"2024-09-15T03:41:13.830529+0000","last_fullsized":"2024-09-15T03:41:13.830529+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'8386","last_scrub_stamp":"2024-09-15T03:41:13.830292+0000","last_deep_scrub":"24'8386","last_deep_scrub_stamp":"2024-09-15T03:41:13.830292+0000","last_clean_scrub_stamp":"2024-09-15T03:41:13.830292+0000","objects_scrubbed":32,"log_size":8386,"log_dups_size":0,"ondisk_log_size":8386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:14:26.084803+0000","scrub_duration":0.46468453999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":126026248,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":36,"num_read_kb":28,"num_write":8349,"num_write_kb":200158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":3570163960,"num_objects":878,"num_object_clones":0,"num_object_copies":1756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":878,"num_whiteouts":0,"num_read":2563,"num_read_kb":2579,"num_write":159019,"num_write_kb":5633737,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":6842679296,"data_stored":6838412408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161653,"ondisk_log_size":161653,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542206,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1361776,"kb_used_data":1047784,"kb_used_omap":1,"kb_used_meta":313982,"kb_avail":93010064,"statfs":{"total":96636764160,"available":95242305536,"internally_reserved":0,"allocated":1072930816,"data_stored":1072149589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":321517912},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 03:40:48 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":1.5880000000000001,"15min":2.444},"min":{"1min":0.42099999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.262,"5min":35.189,"15min":35.189},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.77400000000000002,"15min":0.75},"min":{"1min":0.48999999999999999,"5min":0.245,"15min":0.245},"max":{"1min":1.2709999999999999,"5min":1.427,"15min":1.427},"last":1.1120000000000001}]},{"osd":1,"last update":"Sun Sep 15 03:40:48 2024","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":1.6299999999999999,"15min":2.4159999999999999},"min":{"1min":0.55700000000000005,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.1779999999999999,"5min":35.223999999999997,"15min":35.223999999999997},"last":1.194},{"interface":"front","average":{"1min":0.88,"5min":0.81100000000000005,"15min":0.77500000000000002},"min":{"1min":0.70399999999999996,"5min":0.308,"15min":0.308},"max":{"1min":1.1619999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":1.069}]},{"osd":2,"last update":"Sun Sep 15 03:40:48 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.76500000000000001,"15min":0.73999999999999999},"min":{"1min":0.63600000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.196,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.728,"15min":2.5150000000000001},"min":{"1min":0.79300000000000004,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.1990000000000001,"5min":35.283000000000001,"15min":35.283000000000001},"last":0.83299999999999996}]},{"osd":3,"last update":"Sun Sep 15 03:40:48 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":0.84499999999999997,"15min":0.76100000000000001},"min":{"1min":0.64700000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.3360000000000001,"5min":1.71,"15min":1.71},"last":0.98199999999999998},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.8080000000000001,"15min":2.5699999999999998},"min":{"1min":0.88800000000000001,"5min":0.378,"15min":0.378},"max":{"1min":1.369,"5min":35.255000000000003,"15min":35.255000000000003},"last":0.79800000000000004}]},{"osd":4,"last update":"Sun Sep 15 03:40:48 2024","interfaces":[{"interface":"back","average":{"1min":1.002,"5min":1.111,"15min":0.82899999999999996},"min":{"1min":0.69299999999999995,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.474,"5min":25.52,"15min":25.52},"last":0.627},{"interface":"front","average":{"1min":1.024,"5min":0.871,"15min":0.79100000000000004},"min":{"1min":0.68899999999999995,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.2809999999999999,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.75600000000000001}]},{"osd":5,"last update":"Sun Sep 15 03:40:48 2024","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.8500000000000001,"15min":2.6480000000000001},"min":{"1min":0.78800000000000003,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.343,"5min":35.070999999999998,"15min":35.070999999999998},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.101,"15min":1.107},"min":{"1min":1.075,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.421,"5min":5.2999999999999998,"15min":5.2999999999999998},"last":0.92800000000000005}]},{"osd":6,"last update":"Sun Sep 15 03:40:48 2024","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.024,"15min":0.90400000000000003},"min":{"1min":1.085,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.429,"5min":2.5449999999999999,"15min":2.5449999999999999},"last":1.028},{"interface":"front","average":{"1min":1.151,"5min":0.97299999999999998,"15min":0.86899999999999999},"min":{"1min":0.71499999999999997,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.452,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":0.95999999999999996}]}]},{"osd":6,"up_from":14,"seq":60129542205,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":651348,"kb_used_data":349836,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93720492,"statfs":{"total":96636764160,"available":95969783808,"internally_reserved":0,"allocated":358232064,"data_stored":357835405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 03:40:43 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":1.0049999999999999,"15min":1.155},"min":{"1min":0.253,"5min":0.25,"15min":0.25},"max":{"1min":1.208,"5min":7.3170000000000002,"15min":7.3170000000000002},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.82399999999999995,"15min":0.77300000000000002},"min":{"1min":0.47799999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.212,"5min":7.3849999999999998,"15min":7.3849999999999998},"last":0.92400000000000004}]},{"osd":1,"last update":"Sun Sep 15 03:40:43 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.72899999999999998,"15min":0.69099999999999995},"min":{"1min":0.30299999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.157,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":1.0189999999999999},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.89600000000000002,"15min":0.91800000000000004},"min":{"1min":0.45800000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.304,"5min":7.3559999999999999,"15min":7.3559999999999999},"last":0.88400000000000001}]},{"osd":2,"last update":"Sun Sep 15 03:40:43 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.92800000000000005,"15min":0.93500000000000005},"min":{"1min":0.436,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.286,"5min":7.4960000000000004,"15min":7.4960000000000004},"last":1.05},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.84699999999999998,"15min":0.88600000000000001},"min":{"1min":0.49199999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.1499999999999999,"5min":1.575,"15min":1.575},"last":0.98799999999999999}]},{"osd":3,"last update":"Sun Sep 15 03:40:43 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":1.0569999999999999,"15min":1.1499999999999999},"min":{"1min":0.42399999999999999,"5min":0.308,"15min":0.308},"max":{"1min":1.236,"5min":7.5869999999999997,"15min":7.5869999999999997},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.2869999999999999,"15min":1.526},"min":{"1min":0.53800000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.423,"5min":11.494,"15min":11.494},"last":1.1359999999999999}]},{"osd":4,"last update":"Sun Sep 15 03:40:43 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":1.0449999999999999,"15min":1.0820000000000001},"min":{"1min":0.32600000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.319,"5min":10.348000000000001,"15min":10.348000000000001},"last":1.274},{"interface":"front","average":{"1min":1.083,"5min":1.0509999999999999,"15min":1.0049999999999999},"min":{"1min":0.44,"5min":0.316,"15min":0.316},"max":{"1min":1.373,"5min":7.5620000000000003,"15min":7.5620000000000003},"last":1.228}]},{"osd":5,"last update":"Sun Sep 15 03:40:43 2024","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.86899999999999999,"15min":0.83799999999999997},"min":{"1min":0.54500000000000004,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.3400000000000001,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":1.097},{"interface":"front","average":{"1min":1.147,"5min":1.0149999999999999,"15min":0.96699999999999997},"min":{"1min":0.56699999999999995,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.444,"5min":1.623,"15min":1.623},"last":1.298}]},{"osd":7,"last update":"Sun Sep 15 03:40:43 2024","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.121,"15min":1.0149999999999999},"min":{"1min":0.40500000000000003,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.4279999999999999,"5min":10.923999999999999,"15min":10.923999999999999},"last":1.198},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.1160000000000001,"15min":1.01},"min":{"1min":0.52100000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.7130000000000001,"5min":10.74,"15min":10.74},"last":1.335}]}]},{"osd":5,"up_from":14,"seq":60129542205,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1179932,"kb_used_data":874260,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":93191908,"statfs":{"total":96636764160,"available":95428513792,"internally_reserved":0,"allocated":895242240,"data_stored":894489264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 03:40:46 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.69699999999999995,"15min":0.66300000000000003},"min":{"1min":0.316,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.0049999999999999,"5min":1.3,"15min":1.3},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.80600000000000005,"15min":0.83899999999999997},"min":{"1min":0.36099999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.3300000000000001,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":0.51300000000000001}]},{"osd":1,"last update":"Sun Sep 15 03:40:46 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.78200000000000003,"15min":0.71599999999999997},"min":{"1min":0.33200000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.2450000000000001,"5min":2.302,"15min":2.302},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.73399999999999999,"15min":0.69599999999999995},"min":{"1min":0.36099999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.222,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.63100000000000001}]},{"osd":2,"last update":"Sun Sep 15 03:40:46 2024","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.85399999999999998,"15min":0.83499999999999996},"min":{"1min":0.48199999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":2.5779999999999998,"5min":2.5779999999999998,"15min":2.5779999999999998},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.85499999999999998,"15min":0.85999999999999999},"min":{"1min":0.376,"5min":0.314,"15min":0.314},"max":{"1min":1.399,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":1.0740000000000001}]},{"osd":3,"last update":"Sun Sep 15 03:40:46 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":1.401,"15min":1.9419999999999999},"min":{"1min":0.438,"5min":0.375,"15min":0.375},"max":{"1min":1.2789999999999999,"5min":26.664999999999999,"15min":26.664999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.96999999999999997,"5min":1.538,"15min":2.1230000000000002},"min":{"1min":0.40500000000000003,"5min":0.312,"15min":0.312},"max":{"1min":1.327,"5min":26.748000000000001,"15min":26.748000000000001},"last":1.153}]},{"osd":4,"last update":"Sun Sep 15 03:40:46 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.90600000000000003,"15min":0.88200000000000001},"min":{"1min":0.40600000000000003,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.3280000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.92800000000000005,"15min":0.88200000000000001},"min":{"1min":0.51100000000000001,"5min":0.318,"15min":0.318},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":0.93899999999999995}]},{"osd":6,"last update":"Sun Sep 15 03:40:46 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.97699999999999998,"15min":0.96099999999999997},"min":{"1min":0.52900000000000003,"5min":0.373,"15min":0.373},"max":{"1min":1.3320000000000001,"5min":1.9730000000000001,"15min":1.9730000000000001},"last":0.98699999999999999},{"interface":"front","average":{"1min":1.056,"5min":1.04,"15min":1.004},"min":{"1min":0.436,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.452,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":0.90200000000000002}]},{"osd":7,"last update":"Sun Sep 15 03:40:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":0.97999999999999998,"15min":0.91400000000000003},"min":{"1min":0.47899999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":1.1000000000000001},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.121,"15min":1.077},"min":{"1min":0.55500000000000005,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.6240000000000001,"5min":4.3070000000000004,"15min":4.3070000000000004},"last":1.0369999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542205,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1279280,"kb_used_data":981928,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93092560,"statfs":{"total":96636764160,"available":95326781440,"internally_reserved":0,"allocated":1005494272,"data_stored":1004659645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 03:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.63900000000000001,"15min":0.69699999999999995},"min":{"1min":0.28199999999999997,"5min":0.22,"15min":0.22},"max":{"1min":0.93200000000000005,"5min":4.7809999999999997,"15min":4.7809999999999997},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.79300000000000004,"15min":0.96899999999999997},"min":{"1min":0.29199999999999998,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":1.2609999999999999,"5min":7.7370000000000001,"15min":7.7370000000000001},"last":0.91800000000000004}]},{"osd":1,"last update":"Sun Sep 15 03:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.77900000000000003,"15min":0.63800000000000001},"min":{"1min":0.29099999999999998,"5min":0.217,"15min":0.217},"max":{"1min":1.032,"5min":13.749000000000001,"15min":13.749000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.83499999999999996,"5min":1.296,"15min":0.96799999999999997},"min":{"1min":0.44500000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.2989999999999999,"5min":46.225999999999999,"15min":46.225999999999999},"last":0.88500000000000001}]},{"osd":2,"last update":"Sun Sep 15 03:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.75700000000000001,"15min":0.71299999999999997},"min":{"1min":0.42399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.88,"5min":0.76200000000000001,"15min":0.72599999999999998},"min":{"1min":0.46899999999999997,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.167,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":1.0249999999999999}]},{"osd":4,"last update":"Sun Sep 15 03:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.93100000000000005,"15min":0.94299999999999995},"min":{"1min":0.56299999999999994,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.5660000000000001,"5min":2.9870000000000001,"15min":2.9870000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":1.016,"5min":0.95099999999999996,"15min":0.96799999999999997},"min":{"1min":0.38500000000000001,"5min":0.376,"15min":0.376},"max":{"1min":1.3380000000000001,"5min":2.609,"15min":2.609},"last":0.83199999999999996}]},{"osd":5,"last update":"Sun Sep 15 03:40:45 2024","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":0.90900000000000003,"15min":0.85499999999999998},"min":{"1min":0.42499999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.6140000000000001,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.155,"5min":1.071,"15min":1.1060000000000001},"min":{"1min":0.498,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.6679999999999999,"5min":5.0309999999999997,"15min":5.0309999999999997},"last":0.97199999999999998}]},{"osd":6,"last update":"Sun Sep 15 03:40:45 2024","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.014,"15min":0.98599999999999999},"min":{"1min":0.59899999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.091,"5min":1.002,"15min":0.97199999999999998},"min":{"1min":0.61099999999999999,"5min":0.503,"15min":0.503},"max":{"1min":1.476,"5min":2.5550000000000002,"15min":2.5550000000000002},"last":1.0840000000000001}]},{"osd":7,"last update":"Sun Sep 15 03:40:45 2024","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.129,"15min":1.1830000000000001},"min":{"1min":0.74099999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.587,"5min":5.5439999999999996,"15min":5.5439999999999996},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.2989999999999999,"15min":1.5269999999999999},"min":{"1min":0.71899999999999997,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.5529999999999999,"5min":9.5850000000000009,"15min":9.5850000000000009},"last":1.111}]}]},{"osd":2,"up_from":14,"seq":60129542206,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":687248,"kb_used_data":377416,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":93684592,"statfs":{"total":96636764160,"available":95933022208,"internally_reserved":0,"allocated":386473984,"data_stored":386097365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 03:40:46 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.59699999999999998,"15min":0.61199999999999999},"min":{"1min":0.39800000000000002,"5min":0.193,"15min":0.193},"max":{"1min":0.96499999999999997,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.66700000000000004,"15min":0.65700000000000003},"min":{"1min":0.54000000000000004,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":1.228,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.68400000000000005}]},{"osd":1,"last update":"Sun Sep 15 03:40:46 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.73099999999999998,"15min":0.72599999999999998},"min":{"1min":0.47299999999999998,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.044,"5min":1.258,"15min":1.258},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.61599999999999999,"15min":0.57999999999999996},"min":{"1min":0.41999999999999998,"5min":0.222,"15min":0.222},"max":{"1min":0.92800000000000005,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.63700000000000001}]},{"osd":3,"last update":"Sun Sep 15 03:40:46 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.68300000000000005,"15min":0.66000000000000003},"min":{"1min":0.42399999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.98299999999999998,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.77800000000000002,"15min":0.73399999999999999},"min":{"1min":0.40100000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":1.075}]},{"osd":4,"last update":"Sun Sep 15 03:40:46 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.84099999999999997,"15min":0.77900000000000003},"min":{"1min":0.55500000000000005,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":0.84799999999999998},{"interface":"front","average":{"1min":1.02,"5min":0.91100000000000003,"15min":0.86599999999999999},"min":{"1min":0.53100000000000003,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.4239999999999999,"5min":1.573,"15min":1.573},"last":1.1559999999999999}]},{"osd":5,"last update":"Sun Sep 15 03:40:46 2024","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.94299999999999995,"15min":0.95399999999999996},"min":{"1min":0.68500000000000005,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.3680000000000001,"5min":4.5839999999999996,"15min":4.5839999999999996},"last":1.0369999999999999},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0549999999999999,"15min":1.113},"min":{"1min":0.50900000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.341,"5min":4.8230000000000004,"15min":4.8230000000000004},"last":1.006}]},{"osd":6,"last update":"Sun Sep 15 03:40:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":0.94199999999999995,"15min":0.86799999999999999},"min":{"1min":0.81100000000000005,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.288,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":0.88400000000000001},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.03,"15min":0.96999999999999997},"min":{"1min":0.49299999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.425,"5min":1.9039999999999999,"15min":1.9039999999999999},"last":1.2090000000000001}]},{"osd":7,"last update":"Sun Sep 15 03:40:46 2024","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.0389999999999999,"15min":1.044},"min":{"1min":0.72599999999999998,"5min":0.379,"15min":0.379},"max":{"1min":1.4590000000000001,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":1.105},{"interface":"front","average":{"1min":1.169,"5min":1.3540000000000001,"15min":1.6080000000000001},"min":{"1min":0.74099999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.4770000000000001,"5min":10.428000000000001,"15min":10.428000000000001},"last":1.2549999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574910,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1618136,"kb_used_data":1320784,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":92753704,"statfs":{"total":96636764160,"available":94979792896,"internally_reserved":0,"allocated":1352482816,"data_stored":1351527913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 03:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.85799999999999998,"15min":0.93999999999999995},"min":{"1min":0.311,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.137,"5min":6.5010000000000003,"15min":6.5010000000000003},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.85299999999999998,"15min":0.89300000000000002},"min":{"1min":0.34300000000000003,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.159,"5min":3.645,"15min":3.645},"last":1.0569999999999999}]},{"osd":1,"last update":"Sun Sep 15 03:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.84499999999999997,"15min":0.88900000000000001},"min":{"1min":0.29299999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.2589999999999999,"5min":3.7069999999999999,"15min":3.7069999999999999},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.81699999999999995,"15min":0.80200000000000005},"min":{"1min":0.44600000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.272,"5min":2.5779999999999998,"15min":2.5779999999999998},"last":0.85899999999999999}]},{"osd":2,"last update":"Sun Sep 15 03:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.84999999999999998,"15min":0.81100000000000005},"min":{"1min":0.45000000000000001,"5min":0.311,"15min":0.311},"max":{"1min":1.212,"5min":2.766,"15min":2.766},"last":1.1479999999999999},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.84199999999999997,"15min":0.77200000000000002},"min":{"1min":0.375,"5min":0.316,"15min":0.316},"max":{"1min":1.256,"5min":3.2749999999999999,"15min":3.2749999999999999},"last":0.95799999999999996}]},{"osd":3,"last update":"Sun Sep 15 03:40:45 2024","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.0609999999999999,"15min":1.173},"min":{"1min":0.501,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.357,"5min":6.3620000000000001,"15min":6.3620000000000001},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.0640000000000001,"5min":0.94899999999999995,"15min":0.87},"min":{"1min":0.58799999999999997,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.4330000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":1.1930000000000001}]},{"osd":5,"last update":"Sun Sep 15 03:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":1.0069999999999999,"15min":1.0629999999999999},"min":{"1min":0.41899999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.6890000000000001,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":0.59799999999999998},{"interface":"front","average":{"1min":1.171,"5min":1.143,"15min":1.107},"min":{"1min":0.53000000000000003,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.651,"5min":4.1710000000000003,"15min":4.1710000000000003},"last":0.98599999999999999}]},{"osd":6,"last update":"Sun Sep 15 03:40:45 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.0309999999999999,"15min":1.0389999999999999},"min":{"1min":0.45800000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.6120000000000001,"5min":3.669,"15min":3.669},"last":0.68899999999999995},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.0609999999999999,"15min":0.96399999999999997},"min":{"1min":0.73799999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.581,"5min":2.5329999999999999,"15min":2.5329999999999999},"last":1.0980000000000001}]},{"osd":7,"last update":"Sun Sep 15 03:40:45 2024","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.018,"15min":0.92700000000000005},"min":{"1min":0.56599999999999995,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.498,"5min":3.238,"15min":3.238},"last":1.016},{"interface":"front","average":{"1min":1.244,"5min":1.1299999999999999,"15min":1.109},"min":{"1min":0.68700000000000006,"5min":0.378,"15min":0.378},"max":{"1min":1.552,"5min":3.8260000000000001,"15min":3.8260000000000001},"last":1.042}]}]},{"osd":1,"up_from":13,"seq":55834574909,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":986296,"kb_used_data":676464,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":93385544,"statfs":{"total":96636764160,"available":95626797056,"internally_reserved":0,"allocated":692699136,"data_stored":692171708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 03:40:47 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.58699999999999997,"15min":0.55100000000000005},"min":{"1min":0.27600000000000002,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.874,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.65900000000000003,"15min":0.60499999999999998},"min":{"1min":0.22900000000000001,"5min":0.187,"15min":0.187},"max":{"1min":0.91000000000000003,"5min":6.6050000000000004,"15min":6.6050000000000004},"last":0.70699999999999996}]},{"osd":2,"last update":"Sun Sep 15 03:40:47 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.64100000000000001,"15min":0.63100000000000001},"min":{"1min":0.314,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.88700000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.745},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.77900000000000003,"15min":0.751},"min":{"1min":0.32000000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.228,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":0.88}]},{"osd":3,"last update":"Sun Sep 15 03:40:47 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.70599999999999996,"15min":0.64600000000000002},"min":{"1min":0.373,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":1.04},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.85599999999999998,"15min":0.74199999999999999},"min":{"1min":0.40300000000000002,"5min":0.255,"15min":0.255},"max":{"1min":1.1659999999999999,"5min":5.6950000000000003,"15min":5.6950000000000003},"last":1.1080000000000001}]},{"osd":4,"last update":"Sun Sep 15 03:40:47 2024","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.0409999999999999,"15min":0.90400000000000003},"min":{"1min":0.46000000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.3959999999999999,"5min":9.1080000000000005,"15min":9.1080000000000005},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.149,"5min":1.1120000000000001,"15min":0.98799999999999999},"min":{"1min":0.59299999999999997,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.5660000000000001,"5min":9.1440000000000001,"15min":9.1440000000000001},"last":1.218}]},{"osd":5,"last update":"Sun Sep 15 03:40:47 2024","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":0.93799999999999994,"15min":0.90100000000000002},"min":{"1min":0.47499999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.5349999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.83999999999999997},{"interface":"front","average":{"1min":1.151,"5min":1.0289999999999999,"15min":0.97499999999999998},"min":{"1min":0.49099999999999999,"5min":0.314,"15min":0.314},"max":{"1min":1.4690000000000001,"5min":2.3580000000000001,"15min":2.3580000000000001},"last":1.306}]},{"osd":6,"last update":"Sun Sep 15 03:40:47 2024","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.0029999999999999,"15min":0.94099999999999995},"min":{"1min":0.436,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.387,"5min":6.4109999999999996,"15min":6.4109999999999996},"last":1.1930000000000001},{"interface":"front","average":{"1min":1.087,"5min":1.0489999999999999,"15min":1.006},"min":{"1min":0.53600000000000003,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.5229999999999999,"5min":5.3170000000000002,"15min":5.3170000000000002},"last":0.98199999999999998}]},{"osd":7,"last update":"Sun Sep 15 03:40:47 2024","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.268,"15min":1.4279999999999999},"min":{"1min":0.55500000000000005,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.522,"5min":12.821,"15min":12.821},"last":1.278},{"interface":"front","average":{"1min":1.137,"5min":1.105,"15min":1.073},"min":{"1min":0.59099999999999997,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.498,"5min":5.2530000000000001,"15min":5.2530000000000001},"last":1.0629999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574910,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1362504,"kb_used_data":1056832,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":93009336,"statfs":{"total":96636764160,"available":95241560064,"internally_reserved":0,"allocated":1082195968,"data_stored":1081279613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Sep 15 03:40:41 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.59399999999999997,"15min":0.59999999999999998},"min":{"1min":0.20799999999999999,"5min":0.19,"15min":0.19},"max":{"1min":2.9279999999999999,"5min":2.9279999999999999,"15min":2.9279999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.58099999999999996,"15min":0.54400000000000004},"min":{"1min":0.41399999999999998,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.53700000000000003}]},{"osd":2,"last update":"Sun Sep 15 03:40:47 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.61899999999999999,"15min":0.55900000000000005},"min":{"1min":0.30399999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.69499999999999995,"15min":0.69899999999999995},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.1479999999999999,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":0.84699999999999998}]},{"osd":3,"last update":"Sun Sep 15 03:40:47 2024","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.73699999999999999,"15min":0.72999999999999998},"min":{"1min":0.47499999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":2.8860000000000001,"5min":2.8860000000000001,"15min":2.8860000000000001},"last":1.002},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.69799999999999995,"15min":0.65800000000000003},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.90600000000000003}]},{"osd":4,"last update":"Sun Sep 15 03:40:41 2024","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.83099999999999996,"15min":0.80200000000000005},"min":{"1min":0.45900000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.1919999999999999,"5min":2.403,"15min":2.403},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.135,"5min":0.89700000000000002,"15min":0.83499999999999996},"min":{"1min":0.41299999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":3.1059999999999999,"5min":3.1059999999999999,"15min":3.1059999999999999},"last":0.97899999999999998}]},{"osd":5,"last update":"Sun Sep 15 03:40:47 2024","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":0.93000000000000005,"15min":0.84399999999999997},"min":{"1min":0.68899999999999995,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":2.9609999999999999,"5min":2.9609999999999999,"15min":2.9609999999999999},"last":1.113},{"interface":"front","average":{"1min":1.1000000000000001,"5min":0.96599999999999997,"15min":0.88400000000000001},"min":{"1min":0.51200000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.54,"5min":2.4319999999999999,"15min":2.4319999999999999},"last":1.1419999999999999}]},{"osd":6,"last update":"Sun Sep 15 03:40:47 2024","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":0.92100000000000004,"15min":0.871},"min":{"1min":0.67500000000000004,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.3580000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.0329999999999999,"15min":1.0089999999999999},"min":{"1min":0.56899999999999995,"5min":0.442,"15min":0.442},"max":{"1min":1.639,"5min":2.5459999999999998,"15min":2.5459999999999998},"last":1.052}]},{"osd":7,"last update":"Sun Sep 15 03:40:47 2024","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.206,"15min":1.2989999999999999},"min":{"1min":0.54300000000000004,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.466,"5min":10.561999999999999,"15min":10.561999999999999},"last":1.0900000000000001},{"interface":"front","average":{"1min":1.254,"5min":0.98899999999999999,"15min":0.91100000000000003},"min":{"1min":0.64200000000000002,"5min":0.377,"15min":0.377},"max":{"1min":3.2120000000000002,"5min":3.2120000000000002,"15min":3.2120000000000002},"last":1.169}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1081413632,"data_stored":1080658160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":692445184,"data_stored":692079164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":386224128,"data_stored":386004840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1005244416,"data_stored":1004567120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1352232960,"data_stored":1351435388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":894988288,"data_stored":894396720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":357982208,"data_stored":357742880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1072148480,"data_stored":1071528136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T03:41:36.142 DEBUG:teuthology.orchestra.run.smithi110:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-15T03:41:36.251 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.246+0000 7f7d0f0e4700 1 Processor -- start 2024-09-15T03:41:36.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.246+0000 7f7d0f0e4700 1 -- start start 2024-09-15T03:41:36.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.246+0000 7f7d0f0e4700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f7d08141a80 0x7f7d0806e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:41:36.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.246+0000 7f7d0f0e4700 1 --2- >> v2:172.21.15.110:3300/0 conn(0x7f7d08141e90 0x7f7d0813cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:41:36.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.246+0000 7f7d0f0e4700 1 -- --> v1:172.21.15.110:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d081415f0 con 0x7f7d080b50e0 2024-09-15T03:41:36.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.246+0000 7f7d0f0e4700 1 -- --> v2:172.21.15.110:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d080b54b0 con 0x7f7d08141e90 2024-09-15T03:41:36.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.246+0000 7f7d0f0e4700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d081411f0 con 0x7f7d08141a80 2024-09-15T03:41:36.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.246+0000 7f7d0d8e1700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f7d08141a80 0x7f7d0806e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:41:36.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.246+0000 7f7d0d8e1700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f7d08141a80 0x7f7d0806e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3301/0 says I am v2:172.21.15.110:38324/0 (socket says 172.21.15.110:38324) 2024-09-15T03:41:36.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.246+0000 7f7d0d8e1700 1 -- 172.21.15.110:0/3664193773 learned_addr learned my addr 172.21.15.110:0/3664193773 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:41:36.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.246+0000 7f7d0e8e3700 1 --2- 172.21.15.110:0/3664193773 >> v2:172.21.15.110:3300/0 conn(0x7f7d08141e90 0x7f7d0813cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:41:36.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.246+0000 7f7d0d8e1700 1 -- 172.21.15.110:0/3664193773 >> v2:172.21.15.110:3300/0 conn(0x7f7d08141e90 msgr2=0x7f7d0813cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:41:36.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.246+0000 7f7d0d8e1700 1 --2- 172.21.15.110:0/3664193773 >> v2:172.21.15.110:3300/0 conn(0x7f7d08141e90 0x7f7d0813cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:41:36.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.246+0000 7f7d0d0e0700 1 -- 172.21.15.110:0/3664193773 <== mon.0 v1:172.21.15.110:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1507442520 0 0) 0x7f7d081415f0 con 0x7f7d080b50e0 2024-09-15T03:41:36.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.246+0000 7f7d0d8e1700 1 -- 172.21.15.110:0/3664193773 >> v1:172.21.15.110:6789/0 conn(0x7f7d080b50e0 legacy=0x7f7d0813dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:41:36.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.246+0000 7f7d0d8e1700 1 -- 172.21.15.110:0/3664193773 --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d0806ed20 con 0x7f7d08141a80 2024-09-15T03:41:36.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.250+0000 7f7d0d8e1700 1 --2- 172.21.15.110:0/3664193773 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f7d08141a80 0x7f7d0806e7e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7d040099b0 tx=0x7f7d04009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=687231d3a0835504 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:41:36.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.250+0000 7f7d0d0e0700 1 -- 172.21.15.110:0/3664193773 <== mon.2 v2:172.21.15.110:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d04009ea0 con 0x7f7d08141a80 2024-09-15T03:41:36.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.250+0000 7f7d0d0e0700 1 -- 172.21.15.110:0/3664193773 <== mon.2 v2:172.21.15.110:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d0400b860 con 0x7f7d08141a80 2024-09-15T03:41:36.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.250+0000 7f7d0d0e0700 1 -- 172.21.15.110:0/3664193773 <== mon.2 v2:172.21.15.110:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0400bc00 con 0x7f7d08141a80 2024-09-15T03:41:36.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.250+0000 7f7d0f0e4700 1 -- 172.21.15.110:0/3664193773 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f7d08141a80 msgr2=0x7f7d0806e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:41:36.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.250+0000 7f7d0f0e4700 1 --2- 172.21.15.110:0/3664193773 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f7d08141a80 0x7f7d0806e7e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7d040099b0 tx=0x7f7d04009250 comp rx=0 tx=0).stop 2024-09-15T03:41:36.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.250+0000 7f7d0f0e4700 1 -- 172.21.15.110:0/3664193773 shutdown_connections 2024-09-15T03:41:36.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.250+0000 7f7d0f0e4700 1 --2- 172.21.15.110:0/3664193773 >> v2:172.21.15.110:3300/0 conn(0x7f7d08141e90 0x7f7d0813cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:41:36.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.250+0000 7f7d0f0e4700 1 --2- 172.21.15.110:0/3664193773 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f7d08141a80 0x7f7d0806e7e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:41:36.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.250+0000 7f7d0f0e4700 1 -- 172.21.15.110:0/3664193773 >> 172.21.15.110:0/3664193773 conn(0x7f7d080680f0 msgr2=0x7f7d080b47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:41:36.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.250+0000 7f7d0f0e4700 1 -- 172.21.15.110:0/3664193773 shutdown_connections 2024-09-15T03:41:36.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.250+0000 7f7d0f0e4700 1 -- 172.21.15.110:0/3664193773 wait complete. 2024-09-15T03:41:36.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.250+0000 7f7d0f0e4700 1 Processor -- start 2024-09-15T03:41:36.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.250+0000 7f7d0f0e4700 1 -- start start 2024-09-15T03:41:36.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.250+0000 7f7d0f0e4700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7d080b50e0 0x7f7d080a9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:41:36.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.250+0000 7f7d0f0e4700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f7d08141a80 0x7f7d08150f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:41:36.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.250+0000 7f7d0f0e4700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7d08141e90 0x7f7d08151460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:41:36.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.250+0000 7f7d0f0e4700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d080b2210 con 0x7f7d080b50e0 2024-09-15T03:41:36.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.250+0000 7f7d0f0e4700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d080ab3d0 con 0x7f7d08141e90 2024-09-15T03:41:36.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.250+0000 7f7d0f0e4700 1 -- --> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d080ab0f0 con 0x7f7d08141a80 2024-09-15T03:41:36.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.250+0000 7f7d0e0e2700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7d080b50e0 0x7f7d080a9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:41:36.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.250+0000 7f7d0d8e1700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f7d08141a80 0x7f7d08150f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:41:36.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.250+0000 7f7d0d8e1700 1 --2- >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f7d08141a80 0x7f7d08150f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3301/0 says I am v2:172.21.15.110:38326/0 (socket says 172.21.15.110:38326) 2024-09-15T03:41:36.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.250+0000 7f7d0d8e1700 1 -- 172.21.15.110:0/2980399924 learned_addr learned my addr 172.21.15.110:0/2980399924 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-15T03:41:36.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.250+0000 7f7d0e0e2700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7d080b50e0 0x7f7d080a9440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37430/0 (socket says 172.21.15.110:37430) 2024-09-15T03:41:36.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.250+0000 7f7d0e8e3700 1 --2- 172.21.15.110:0/2980399924 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7d08141e90 0x7f7d08151460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:41:36.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.254+0000 7f7d0e0e2700 1 -- 172.21.15.110:0/2980399924 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f7d08141a80 msgr2=0x7f7d08150f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:41:36.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.254+0000 7f7d0e0e2700 1 --2- 172.21.15.110:0/2980399924 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f7d08141a80 0x7f7d08150f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:41:36.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.254+0000 7f7d0e0e2700 1 -- 172.21.15.110:0/2980399924 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7d08141e90 msgr2=0x7f7d08151460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:41:36.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.254+0000 7f7d0e0e2700 1 --2- 172.21.15.110:0/2980399924 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7d08141e90 0x7f7d08151460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:41:36.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.254+0000 7f7d0e0e2700 1 -- 172.21.15.110:0/2980399924 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d040094f0 con 0x7f7d080b50e0 2024-09-15T03:41:36.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.254+0000 7f7d0e8e3700 1 --2- 172.21.15.110:0/2980399924 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7d08141e90 0x7f7d08151460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T03:41:36.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.254+0000 7f7d0d8e1700 1 --2- 172.21.15.110:0/2980399924 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f7d08141a80 0x7f7d08150f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T03:41:36.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.254+0000 7f7d0e0e2700 1 --2- 172.21.15.110:0/2980399924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7d080b50e0 0x7f7d080a9440 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7cf80011e0 tx=0x7f7cf8007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:41:36.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.254+0000 7f7cfe7fc700 1 -- 172.21.15.110:0/2980399924 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf8011590 con 0x7f7d080b50e0 2024-09-15T03:41:36.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.254+0000 7f7d0f0e4700 1 -- 172.21.15.110:0/2980399924 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d08142730 con 0x7f7d080b50e0 2024-09-15T03:41:36.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.254+0000 7f7cfe7fc700 1 -- 172.21.15.110:0/2980399924 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cf8011900 con 0x7f7d080b50e0 2024-09-15T03:41:36.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.254+0000 7f7d0f0e4700 1 -- 172.21.15.110:0/2980399924 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d08136650 con 0x7f7d080b50e0 2024-09-15T03:41:36.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.254+0000 7f7cfe7fc700 1 -- 172.21.15.110:0/2980399924 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf8011a70 con 0x7f7d080b50e0 2024-09-15T03:41:36.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.254+0000 7f7cfe7fc700 1 -- 172.21.15.110:0/2980399924 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) v1 ==== 85847+0+0 (secure 0 0 0) 0x7f7cf8012420 con 0x7f7d080b50e0 2024-09-15T03:41:36.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.254+0000 7f7d0f0e4700 1 -- 172.21.15.110:0/2980399924 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d081428c0 con 0x7f7d080b50e0 2024-09-15T03:41:36.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.254+0000 7f7cfe7fc700 1 --2- 172.21.15.110:0/2980399924 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f7cec067a80 0x7f7cec069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T03:41:36.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.254+0000 7f7cfe7fc700 1 -- 172.21.15.110:0/2980399924 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5641+0+0 (secure 0 0 0) 0x7f7cf8084aa0 con 0x7f7d080b50e0 2024-09-15T03:41:36.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.254+0000 7f7d0d8e1700 1 --2- 172.21.15.110:0/2980399924 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f7cec067a80 0x7f7cec069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T03:41:36.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.258+0000 7f7d0d8e1700 1 --2- 172.21.15.110:0/2980399924 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f7cec067a80 0x7f7cec069f30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7d04003080 tx=0x7f7d04017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T03:41:36.268 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.262+0000 7f7cfe7fc700 1 -- 172.21.15.110:0/2980399924 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f7cf8051c90 con 0x7f7d080b50e0 2024-09-15T03:41:36.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.430+0000 7f7d0f0e4700 1 -- 172.21.15.110:0/2980399924 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f7d080721b0 con 0x7f7d080b50e0 2024-09-15T03:41:36.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.434+0000 7f7cfe7fc700 1 -- 172.21.15.110:0/2980399924 <== mon.0 v2:172.21.15.110:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f7cf8051c90 con 0x7f7d080b50e0 2024-09-15T03:41:36.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.434+0000 7f7cfe7fc700 1 -- 172.21.15.110:0/2980399924 <== mon.0 v2:172.21.15.110:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v2)=0 v2) v1 ==== 123+0+0 (secure 0 0 0) 0x7f7cf8017070 con 0x7f7d080b50e0 2024-09-15T03:41:36.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.434+0000 7f7d0f0e4700 1 -- 172.21.15.110:0/2980399924 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f7cec067a80 msgr2=0x7f7cec069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:41:36.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.434+0000 7f7d0f0e4700 1 --2- 172.21.15.110:0/2980399924 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f7cec067a80 0x7f7cec069f30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7d04003080 tx=0x7f7d04017000 comp rx=0 tx=0).stop 2024-09-15T03:41:36.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.434+0000 7f7d0f0e4700 1 -- 172.21.15.110:0/2980399924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7d080b50e0 msgr2=0x7f7d080a9440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:41:36.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.434+0000 7f7d0f0e4700 1 --2- 172.21.15.110:0/2980399924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7d080b50e0 0x7f7d080a9440 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7cf80011e0 tx=0x7f7cf8007d30 comp rx=0 tx=0).stop 2024-09-15T03:41:36.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.438+0000 7f7d0f0e4700 1 -- 172.21.15.110:0/2980399924 shutdown_connections 2024-09-15T03:41:36.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.438+0000 7f7d0f0e4700 1 --2- 172.21.15.110:0/2980399924 >> [v2:172.21.15.110:6832/22190,v1:172.21.15.110:6833/22190] conn(0x7f7cec067a80 0x7f7cec069f30 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:41:36.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.438+0000 7f7d0f0e4700 1 --2- 172.21.15.110:0/2980399924 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7d08141e90 0x7f7d08151460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:41:36.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.438+0000 7f7d0f0e4700 1 --2- 172.21.15.110:0/2980399924 >> [v2:172.21.15.110:3301/0,v1:172.21.15.110:6790/0] conn(0x7f7d08141a80 0x7f7d08150f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:41:36.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.438+0000 7f7d0f0e4700 1 --2- 172.21.15.110:0/2980399924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7d080b50e0 0x7f7d080a9440 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T03:41:36.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.438+0000 7f7d0f0e4700 1 -- 172.21.15.110:0/2980399924 >> 172.21.15.110:0/2980399924 conn(0x7f7d080680f0 msgr2=0x7f7d08140060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:41:36.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.438+0000 7f7d0f0e4700 1 -- 172.21.15.110:0/2980399924 shutdown_connections 2024-09-15T03:41:36.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-15T03:41:36.438+0000 7f7d0f0e4700 1 -- 172.21.15.110:0/2980399924 wait complete. 2024-09-15T03:41:36.456 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-15T03:41:36.456 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-15T03:41:36.457 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-15T03:41:36.457 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T03:41:36.556 INFO:tasks.ceph.osd.0:Stopped 2024-09-15T03:41:36.556 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-15T03:41:36.556 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T03:41:36.640 INFO:tasks.ceph.osd.1:Stopped 2024-09-15T03:41:36.640 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-15T03:41:36.640 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T03:41:36.716 INFO:tasks.ceph.osd.2:Stopped 2024-09-15T03:41:36.716 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-15T03:41:36.716 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T03:41:40.213 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-09-15T03:41:42.818 INFO:tasks.ceph.osd.3:Stopped 2024-09-15T03:41:42.818 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-15T03:41:42.818 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T03:41:42.896 INFO:tasks.ceph.osd.4:Stopped 2024-09-15T03:41:42.896 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-15T03:41:42.896 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T03:41:42.994 INFO:tasks.ceph.osd.5:Stopped 2024-09-15T03:41:42.994 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-15T03:41:42.994 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T03:41:43.066 INFO:tasks.ceph.osd.6:Stopped 2024-09-15T03:41:43.067 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-15T03:41:43.067 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T03:41:43.167 INFO:tasks.ceph.osd.7:Stopped 2024-09-15T03:41:43.167 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-15T03:41:43.167 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-15T03:41:43.167 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T03:41:43.198 INFO:tasks.ceph.mgr.y:Stopped 2024-09-15T03:41:43.198 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-15T03:41:43.198 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T03:41:43.230 INFO:tasks.ceph.mgr.x:Stopped 2024-09-15T03:41:43.230 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-15T03:41:43.230 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-15T03:41:43.231 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T03:41:43.259 INFO:tasks.ceph.mon.a:Stopped 2024-09-15T03:41:43.259 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-15T03:41:43.259 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T03:41:43.276 INFO:tasks.ceph.mon.c:Stopped 2024-09-15T03:41:43.276 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-15T03:41:43.277 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T03:41:43.295 INFO:tasks.ceph.mon.b:Stopped 2024-09-15T03:41:43.295 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-15T03:41:43.295 DEBUG:teuthology.orchestra.run.smithi110:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | head -n 1 2024-09-15T03:41:43.314 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi110.front.sepia.ceph.com 2024-09-15T03:41:43.314 DEBUG:teuthology.orchestra.run.smithi110:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-15T03:41:43.608 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi110.front.sepia.ceph.com 2024-09-15T03:41:43.608 DEBUG:teuthology.orchestra.run.smithi110:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-15T03:41:43.880 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi110.front.sepia.ceph.com 2024-09-15T03:41:43.880 DEBUG:teuthology.orchestra.run.smithi110:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-15T03:41:44.068 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi110.front.sepia.ceph.com 2024-09-15T03:41:44.069 DEBUG:teuthology.orchestra.run.smithi110:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-15T03:41:44.279 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi177.front.sepia.ceph.com 2024-09-15T03:41:44.279 DEBUG:teuthology.orchestra.run.smithi177:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-15T03:41:44.533 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi177.front.sepia.ceph.com 2024-09-15T03:41:44.533 DEBUG:teuthology.orchestra.run.smithi177:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-15T03:41:44.832 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi177.front.sepia.ceph.com 2024-09-15T03:41:44.832 DEBUG:teuthology.orchestra.run.smithi177:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-15T03:41:44.997 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi177.front.sepia.ceph.com 2024-09-15T03:41:44.998 DEBUG:teuthology.orchestra.run.smithi177:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-15T03:41:45.194 INFO:tasks.ceph:Archiving mon data... 2024-09-15T03:41:45.195 DEBUG:teuthology.misc:Transferring archived files from smithi110:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905284/data/mon.a.tgz 2024-09-15T03:41:45.195 DEBUG:teuthology.orchestra.run.smithi110:> mktemp 2024-09-15T03:41:45.202 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/tmp.BUcABZFHh8 2024-09-15T03:41:45.202 DEBUG:teuthology.orchestra.run.smithi110:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.BUcABZFHh8 2024-09-15T03:41:45.214 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-09-15T03:41:45.275 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod 0666 /tmp/tmp.BUcABZFHh8 2024-09-15T03:41:45.301 DEBUG:teuthology.orchestra.remote:smithi110:/tmp/tmp.BUcABZFHh8 is 289KB 2024-09-15T03:41:45.363 DEBUG:teuthology.orchestra.run.smithi110:> rm -fr /tmp/tmp.BUcABZFHh8 2024-09-15T03:41:45.370 DEBUG:teuthology.misc:Transferring archived files from smithi110:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905284/data/mon.c.tgz 2024-09-15T03:41:45.370 DEBUG:teuthology.orchestra.run.smithi110:> mktemp 2024-09-15T03:41:45.376 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/tmp.TvgrqwYjUr 2024-09-15T03:41:45.377 DEBUG:teuthology.orchestra.run.smithi110:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.TvgrqwYjUr 2024-09-15T03:41:45.448 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod 0666 /tmp/tmp.TvgrqwYjUr 2024-09-15T03:41:45.516 DEBUG:teuthology.orchestra.remote:smithi110:/tmp/tmp.TvgrqwYjUr is 293KB 2024-09-15T03:41:45.538 DEBUG:teuthology.orchestra.run.smithi110:> rm -fr /tmp/tmp.TvgrqwYjUr 2024-09-15T03:41:45.544 DEBUG:teuthology.misc:Transferring archived files from smithi177:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905284/data/mon.b.tgz 2024-09-15T03:41:45.544 DEBUG:teuthology.orchestra.run.smithi177:> mktemp 2024-09-15T03:41:45.551 INFO:teuthology.orchestra.run.smithi177.stdout:/tmp/tmp.am8omXyNGo 2024-09-15T03:41:45.551 DEBUG:teuthology.orchestra.run.smithi177:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.am8omXyNGo 2024-09-15T03:41:45.626 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 0666 /tmp/tmp.am8omXyNGo 2024-09-15T03:41:45.694 DEBUG:teuthology.orchestra.remote:smithi177:/tmp/tmp.am8omXyNGo is 293KB 2024-09-15T03:41:45.756 DEBUG:teuthology.orchestra.run.smithi177:> rm -fr /tmp/tmp.am8omXyNGo 2024-09-15T03:41:45.762 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-15T03:41:45.762 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-15T03:41:45.764 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-15T03:41:45.780 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-09-15T03:41:45.780 DEBUG:teuthology.orchestra.run.smithi110:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-15T03:41:45.783 DEBUG:teuthology.orchestra.run.smithi177:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-15T03:41:45.804 INFO:teuthology.orchestra.run.smithi110.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-15T03:41:45.809 INFO:teuthology.orchestra.run.smithi177.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-15T03:41:45.816 INFO:tasks.ceph:Archiving crash dumps... 2024-09-15T03:41:45.818 DEBUG:teuthology.misc:Transferring archived files from smithi110:/var/lib/ceph/crash to /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905284/remote/smithi110/crash 2024-09-15T03:41:45.819 DEBUG:teuthology.orchestra.run.smithi110:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-15T03:41:45.868 DEBUG:teuthology.misc:Transferring archived files from smithi177:/var/lib/ceph/crash to /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905284/remote/smithi177/crash 2024-09-15T03:41:45.869 DEBUG:teuthology.orchestra.run.smithi177:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-15T03:41:45.886 INFO:tasks.ceph:Compressing logs... 2024-09-15T03:41:45.886 DEBUG:teuthology.orchestra.run.smithi110:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-15T03:41:45.888 DEBUG:teuthology.orchestra.run.smithi177:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-15T03:41:45.916 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-15T03:41:45.916 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22333.log 2024-09-15T03:41:45.916 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-15T03:41:45.918 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.22333.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-15T03:41:45.918 INFO:teuthology.orchestra.run.smithi177.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.22333.log.gz 2024-09-15T03:41:45.918 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-15T03:41:45.918 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-osd.5.log: gzip /var/log/ceph/ceph-mgr.x.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.22190.log 2024-09-15T03:41:45.918 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22240.log 2024-09-15T03:41:45.920 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.22190.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-15T03:41:45.920 INFO:teuthology.orchestra.run.smithi177.stderr: 88.1% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-15T03:41:45.920 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.22240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22358.log 2024-09-15T03:41:45.920 INFO:teuthology.orchestra.run.smithi177.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.22190.log.gz 2024-09-15T03:41:45.920 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-osd.6.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.22240.log.gz 2024-09-15T03:41:45.921 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27532.log 2024-09-15T03:41:45.922 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.libcephsqlite.27239.log 2024-09-15T03:41:45.922 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-15T03:41:45.922 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-15T03:41:45.924 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28080.log 2024-09-15T03:41:45.924 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.27532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.libcephsqlite.27382.log 2024-09-15T03:41:45.924 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.libcephsqlite.27239.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.27532.log.gz 2024-09-15T03:41:45.925 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.28080.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.17377.log 2024-09-15T03:41:45.925 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26250.log 2024-09-15T03:41:45.925 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.libcephsqlite.27382.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.28080.log.gz 2024-09-15T03:41:45.925 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.22358.log: gzip -5 --verbose /var/log/ceph/ceph.log: -- /var/log/ceph/ceph-client.admin.22215.log 2024-09-15T03:41:45.925 INFO:teuthology.orchestra.run.smithi177.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.22358.log.gz 2024-09-15T03:41:45.925 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-osd.4.log: 84.3% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-15T03:41:45.926 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22281.log 2024-09-15T03:41:45.926 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22308.log 2024-09-15T03:41:45.927 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose /var/log/ceph/ceph.tmp-client.admin.17377.log: -- /var/log/ceph/ceph-mon.a.log 2024-09-15T03:41:45.928 INFO:teuthology.orchestra.run.smithi110.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.17377.log.gz 2024-09-15T03:41:45.928 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26464.log 2024-09-15T03:41:45.928 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.26250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26242.log 2024-09-15T03:41:45.928 INFO:teuthology.orchestra.run.smithi110.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.26250.log.gz 2024-09-15T03:41:45.928 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26442.log 2024-09-15T03:41:45.928 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.26464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27559.log 2024-09-15T03:41:45.928 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.26242.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.26464.log.gz 2024-09-15T03:41:45.931 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.22215.log: /var/log/ceph/ceph-client.admin.22281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22384.log 2024-09-15T03:41:45.932 INFO:teuthology.orchestra.run.smithi177.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.22281.log.gz 2024-09-15T03:41:45.932 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.22308.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-15T03:41:45.932 INFO:teuthology.orchestra.run.smithi177.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.22215.log.gz 2024-09-15T03:41:45.932 INFO:teuthology.orchestra.run.smithi177.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.22308.log.gz 2024-09-15T03:41:45.932 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.22384.log: 93.7% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-15T03:41:45.932 INFO:teuthology.orchestra.run.smithi177.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.22384.log.gz 2024-09-15T03:41:45.935 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26033.log 2024-09-15T03:41:45.935 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.26442.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.26242.log.gz 2024-09-15T03:41:45.935 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28018.log 2024-09-15T03:41:45.935 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.27559.log: /var/log/ceph/ceph-client.admin.26033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27694.log 2024-09-15T03:41:45.935 INFO:teuthology.orchestra.run.smithi110.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.26442.log.gz 2024-09-15T03:41:45.935 INFO:teuthology.orchestra.run.smithi110.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.26033.log.gz 2024-09-15T03:41:45.935 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.28018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27856.log 2024-09-15T03:41:45.935 INFO:teuthology.orchestra.run.smithi110.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.27559.log.gz 2024-09-15T03:41:45.935 INFO:teuthology.orchestra.run.smithi110.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.28018.log.gz 2024-09-15T03:41:45.936 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26455.log 2024-09-15T03:41:45.936 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.27694.log: /var/log/ceph/ceph-client.admin.27856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25832.log 2024-09-15T03:41:45.936 INFO:teuthology.orchestra.run.smithi110.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.27694.log.gz 2024-09-15T03:41:45.936 INFO:teuthology.orchestra.run.smithi110.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.27856.log.gz 2024-09-15T03:41:45.936 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.26455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25952.log 2024-09-15T03:41:45.936 INFO:teuthology.orchestra.run.smithi110.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.26455.log.gz 2024-09-15T03:41:45.936 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.libcephsqlite.27152.log 2024-09-15T03:41:45.936 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.25952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27586.log 2024-09-15T03:41:45.936 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.25832.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.25952.log.gz 2024-09-15T03:41:45.936 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27667.log 2024-09-15T03:41:45.936 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.libcephsqlite.27152.log: /var/log/ceph/ceph-client.admin.27586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26029.log 2024-09-15T03:41:45.937 INFO:teuthology.orchestra.run.smithi110.stderr: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-15T03:41:45.937 INFO:teuthology.orchestra.run.smithi110.stderr: -- replaced with /var/log/ceph/ceph-client.admin.25832.log.gz 2024-09-15T03:41:45.937 INFO:teuthology.orchestra.run.smithi110.stderr: 87.4% 89.6%/var/log/ceph/ceph-client.admin.27667.log: -- replaced with /var/log/ceph/ceph-client.admin.27586.log.gz -- replaced with /var/log/ceph/ceph-client.libcephsqlite.27152.log.gz 2024-09-15T03:41:45.937 INFO:teuthology.orchestra.run.smithi110.stderr: 2024-09-15T03:41:45.937 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.26029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.libcephsqlite.27271.log 2024-09-15T03:41:45.937 INFO:teuthology.orchestra.run.smithi110.stderr: 88.4% 88.6% -- replaced with /var/log/ceph/ceph-client.admin.26029.log.gz 2024-09-15T03:41:45.937 INFO:teuthology.orchestra.run.smithi110.stderr: -- replaced with /var/log/ceph/ceph-client.admin.27667.log.gz 2024-09-15T03:41:45.938 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28053.log 2024-09-15T03:41:45.938 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.libcephsqlite.27271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.libcephsqlite.27195.log 2024-09-15T03:41:45.938 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27451.log 2024-09-15T03:41:45.939 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.28053.log: /var/log/ceph/ceph-client.libcephsqlite.27195.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.28053.log.gz 2024-09-15T03:41:45.939 INFO:teuthology.orchestra.run.smithi110.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.libcephsqlite.27195.log.gz 2024-09-15T03:41:45.939 INFO:teuthology.orchestra.run.smithi110.stderr: 92.7% -- replaced with /var/log/ceph/ceph-client.libcephsqlite.27239.log.gz 2024-09-15T03:41:45.941 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26032.log 2024-09-15T03:41:45.943 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.27451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.libcephsqlite.27340.log 2024-09-15T03:41:45.943 INFO:teuthology.orchestra.run.smithi110.stderr: 88.9%gzip /var/log/ceph/ceph-client.admin.26032.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.27640.log 2024-09-15T03:41:45.943 INFO:teuthology.orchestra.run.smithi110.stderr: -- replaced with /var/log/ceph/ceph-client.admin.27451.log.gz 2024-09-15T03:41:45.943 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.libcephsqlite.27340.log: gzip 88.7%-5 -- replaced with /var/log/ceph/ceph-client.admin.26032.log.gz--verbose 2024-09-15T03:41:45.943 INFO:teuthology.orchestra.run.smithi110.stderr:-- /var/log/ceph/ceph-client.admin.22351.log 2024-09-15T03:41:45.943 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26247.log 2024-09-15T03:41:45.944 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.27640.log: /var/log/ceph/ceph-client.admin.22351.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.27640.log.gz 2024-09-15T03:41:45.944 INFO:teuthology.orchestra.run.smithi110.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.22351.log.gz 2024-09-15T03:41:45.945 INFO:teuthology.orchestra.run.smithi110.stderr: 94.6% -- replaced with /var/log/ceph/ceph-client.libcephsqlite.27271.log.gz 2024-09-15T03:41:45.945 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27964.log 2024-09-15T03:41:45.945 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26247.log: /var/log/ceph/ceph-mon.c.log 2024-09-15T03:41:45.946 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-15T03:41:45.946 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.27964.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.26247.log.gz 2024-09-15T03:41:45.946 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26463.log 2024-09-15T03:41:45.947 INFO:teuthology.orchestra.run.smithi110.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.27964.log.gz 2024-09-15T03:41:45.947 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph.audit.log: 93.2% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-15T03:41:45.949 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26681.log 2024-09-15T03:41:45.950 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27883.log 2024-09-15T03:41:45.950 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.26463.log: /var/log/ceph/ceph-client.admin.26681.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.26463.log.gz 2024-09-15T03:41:45.950 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27721.log 2024-09-15T03:41:45.951 INFO:teuthology.orchestra.run.smithi110.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.26681.log.gz 2024-09-15T03:41:45.951 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.27883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27991.log 2024-09-15T03:41:45.951 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.27721.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.27883.log.gz 2024-09-15T03:41:45.952 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26025.log 2024-09-15T03:41:45.952 INFO:teuthology.orchestra.run.smithi110.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.27721.log.gz 2024-09-15T03:41:45.952 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.27991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27910.log 2024-09-15T03:41:45.953 INFO:teuthology.orchestra.run.smithi110.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.27991.log.gz 2024-09-15T03:41:45.953 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.26025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22493.log 2024-09-15T03:41:45.953 INFO:teuthology.orchestra.run.smithi110.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.26025.log.gz 2024-09-15T03:41:45.953 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.27910.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.27910.log.gz 2024-09-15T03:41:45.954 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26461.log 2024-09-15T03:41:45.954 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.22493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27937.log 2024-09-15T03:41:45.954 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.26461.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.22493.log.gz 2024-09-15T03:41:45.955 INFO:teuthology.orchestra.run.smithi110.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.26461.log.gz 2024-09-15T03:41:45.957 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26761.log 2024-09-15T03:41:45.959 INFO:teuthology.orchestra.run.smithi110.stderr:gzip /var/log/ceph/ceph-client.admin.27937.log: -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-15T03:41:45.959 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.26761.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.27937.log.gz 2024-09-15T03:41:45.959 INFO:teuthology.orchestra.run.smithi110.stderr: 87.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26245.log 2024-09-15T03:41:45.959 INFO:teuthology.orchestra.run.smithi110.stderr: -- replaced with /var/log/ceph/ceph-client.admin.26761.log.gz 2024-09-15T03:41:45.959 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-osd.1.log: 96.4% -- replaced with /var/log/ceph/ceph-client.libcephsqlite.27340.log.gz 2024-09-15T03:41:45.959 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27505.log 2024-09-15T03:41:45.960 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26246.log 2024-09-15T03:41:45.960 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.26245.log: /var/log/ceph/ceph-client.admin.27505.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.26245.log.gz 2024-09-15T03:41:45.961 INFO:teuthology.orchestra.run.smithi110.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.27505.log.gz 2024-09-15T03:41:45.961 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.libcephsqlite.27359.log 2024-09-15T03:41:45.962 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26654.log 2024-09-15T03:41:45.962 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.26246.log: /var/log/ceph/ceph-client.libcephsqlite.27359.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.26246.log.gz 2024-09-15T03:41:45.963 INFO:teuthology.orchestra.run.smithi110.stderr: 90.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.libcephsqlite.26786.log 2024-09-15T03:41:45.963 INFO:teuthology.orchestra.run.smithi110.stderr: -- replaced with /var/log/ceph/ceph-client.libcephsqlite.27359.log.gz 2024-09-15T03:41:45.963 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.26654.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.26654.log.gz 2024-09-15T03:41:45.963 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25806.log 2024-09-15T03:41:45.963 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27748.log 2024-09-15T03:41:45.964 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.libcephsqlite.26786.log: /var/log/ceph/ceph-client.admin.25806.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.25806.log.gz 2024-09-15T03:41:45.965 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26466.log 2024-09-15T03:41:45.966 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.27748.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.27748.log.gz 2024-09-15T03:41:45.966 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27802.log 2024-09-15T03:41:45.967 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.26466.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.26466.log.gz 2024-09-15T03:41:45.967 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26244.log 2024-09-15T03:41:45.968 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.27802.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.27802.log.gz 2024-09-15T03:41:45.968 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26034.log 2024-09-15T03:41:45.969 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.26244.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.26244.log.gz 2024-09-15T03:41:45.969 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26243.log 2024-09-15T03:41:45.970 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.26034.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.26034.log.gz 2024-09-15T03:41:45.970 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-15T03:41:45.970 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.26243.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.26243.log.gz 2024-09-15T03:41:45.971 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27613.log 2024-09-15T03:41:45.973 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph.log: 91.5% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-15T03:41:45.973 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.libcephsqlite.27291.log 2024-09-15T03:41:45.974 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.27613.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.27613.log.gz 2024-09-15T03:41:45.974 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26021.log 2024-09-15T03:41:45.975 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.libcephsqlite.27291.log: 89.6% -- replaced with /var/log/ceph/ceph-client.libcephsqlite.27291.log.gz 2024-09-15T03:41:45.975 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26465.log 2024-09-15T03:41:45.976 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.26021.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.26021.log.gz 2024-09-15T03:41:45.977 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-15T03:41:45.977 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.26465.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.26465.log.gz 2024-09-15T03:41:45.977 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25979.log 2024-09-15T03:41:45.981 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.libcephsqlite.27174.log 2024-09-15T03:41:45.982 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.25979.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.25979.log.gz 2024-09-15T03:41:45.985 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22187.log 2024-09-15T03:41:45.989 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.libcephsqlite.27174.log: 91.8% -- replaced with /var/log/ceph/ceph-client.libcephsqlite.27174.log.gz 2024-09-15T03:41:45.989 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27829.log 2024-09-15T03:41:45.990 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.22187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.libcephsqlite.27314.log 2024-09-15T03:41:45.990 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.27829.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.27829.log.gz 2024-09-15T03:41:45.991 INFO:teuthology.orchestra.run.smithi110.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.22187.log.gz 2024-09-15T03:41:45.992 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26028.log 2024-09-15T03:41:45.995 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.libcephsqlite.27314.log: 91.7% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-15T03:41:45.996 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25893.log 2024-09-15T03:41:45.997 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.26028.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.26028.log.gz 2024-09-15T03:41:45.997 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27478.log 2024-09-15T03:41:45.998 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.25893.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.25893.log.gz 2024-09-15T03:41:45.998 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26708.log 2024-09-15T03:41:45.999 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.27478.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.27478.log.gz 2024-09-15T03:41:45.999 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26031.log 2024-09-15T03:41:46.000 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.26708.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.26708.log.gz 2024-09-15T03:41:46.001 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28105.log 2024-09-15T03:41:46.002 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.26031.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.26031.log.gz 2024-09-15T03:41:46.009 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27775.log 2024-09-15T03:41:46.010 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.28105.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.28105.log.gz 2024-09-15T03:41:46.011 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26441.log 2024-09-15T03:41:46.011 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.27775.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.27775.log.gz 2024-09-15T03:41:46.012 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-15T03:41:46.013 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.26441.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.26441.log.gz 2024-09-15T03:41:46.013 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.libcephsqlite.27422.log 2024-09-15T03:41:46.034 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26249.log 2024-09-15T03:41:46.036 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.libcephsqlite.27422.log: 91.3% -- replaced with /var/log/ceph/ceph-client.libcephsqlite.27422.log.gz 2024-09-15T03:41:46.037 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-osd.7.log: 92.4% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-15T03:41:46.058 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26733.log 2024-09-15T03:41:46.058 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.26249.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.26249.log.gz 2024-09-15T03:41:46.074 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25925.log 2024-09-15T03:41:46.074 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.26733.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.26733.log.gz 2024-09-15T03:41:46.089 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-15T03:41:46.090 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.25925.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.25925.log.gz 2024-09-15T03:41:46.091 INFO:teuthology.orchestra.run.smithi110.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-15T03:41:46.091 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.libcephsqlite.27216.log 2024-09-15T03:41:46.106 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-osd.2.log: /var/log/ceph/ceph-client.libcephsqlite.27216.log: 89.2% -- replaced with /var/log/ceph/ceph-client.libcephsqlite.27216.log.gz 2024-09-15T03:41:46.191 INFO:teuthology.orchestra.run.smithi110.stderr: 95.0% -- replaced with /var/log/ceph/ceph-client.libcephsqlite.27314.log.gz 2024-09-15T03:41:46.194 INFO:teuthology.orchestra.run.smithi110.stderr: 90.8% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-15T03:41:46.211 INFO:teuthology.orchestra.run.smithi110.stderr: 94.9% -- replaced with /var/log/ceph/ceph-client.libcephsqlite.27382.log.gz 2024-09-15T03:41:47.971 INFO:teuthology.orchestra.run.smithi110.stderr: 95.2% -- replaced with /var/log/ceph/ceph-client.libcephsqlite.26786.log.gz 2024-09-15T03:41:49.548 INFO:teuthology.orchestra.run.smithi110.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-15T03:41:50.616 INFO:teuthology.orchestra.run.smithi177.stderr: 92.4% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-15T03:41:55.634 INFO:teuthology.orchestra.run.smithi110.stderr: 92.1% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-15T03:41:57.228 INFO:teuthology.orchestra.run.smithi177.stderr: 92.0% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-15T03:42:00.448 INFO:teuthology.orchestra.run.smithi110.stderr: 91.8% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-15T03:42:00.573 INFO:teuthology.orchestra.run.smithi177.stderr: 91.7% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-15T03:42:00.635 INFO:teuthology.orchestra.run.smithi110.stderr: 91.6% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-15T03:42:00.635 INFO:teuthology.orchestra.run.smithi110.stderr: 2024-09-15T03:42:00.636 INFO:teuthology.orchestra.run.smithi110.stderr:real 0m14.744s 2024-09-15T03:42:00.636 INFO:teuthology.orchestra.run.smithi110.stderr:user 0m43.829s 2024-09-15T03:42:00.636 INFO:teuthology.orchestra.run.smithi110.stderr:sys 0m1.534s 2024-09-15T03:42:01.009 INFO:teuthology.orchestra.run.smithi177.stderr: 91.9% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-15T03:42:01.011 INFO:teuthology.orchestra.run.smithi177.stderr: 2024-09-15T03:42:01.011 INFO:teuthology.orchestra.run.smithi177.stderr:real 0m15.119s 2024-09-15T03:42:01.011 INFO:teuthology.orchestra.run.smithi177.stderr:user 0m44.450s 2024-09-15T03:42:01.011 INFO:teuthology.orchestra.run.smithi177.stderr:sys 0m1.472s 2024-09-15T03:42:01.011 INFO:tasks.ceph:Archiving logs... 2024-09-15T03:42:01.011 DEBUG:teuthology.misc:Transferring archived files from smithi110:/var/log/ceph to /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905284/remote/smithi110/log 2024-09-15T03:42:01.012 DEBUG:teuthology.orchestra.run.smithi110:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-15T03:42:08.410 DEBUG:teuthology.misc:Transferring archived files from smithi177:/var/log/ceph to /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905284/remote/smithi177/log 2024-09-15T03:42:08.411 DEBUG:teuthology.orchestra.run.smithi177:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-15T03:42:16.319 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-15T03:42:16.327 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-15T03:42:16.328 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-15T03:42:16.330 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-15T03:42:16.410 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-15T03:42:16.419 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-15T03:42:16.419 DEBUG:teuthology.orchestra.run.smithi110:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T03:42:16.422 DEBUG:teuthology.orchestra.run.smithi177:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T03:42:16.442 INFO:teuthology.orchestra.run.smithi110.stdout: remote refid st t when poll reach delay offset jitter 2024-09-15T03:42:16.442 INFO:teuthology.orchestra.run.smithi110.stdout:============================================================================== 2024-09-15T03:42:16.442 INFO:teuthology.orchestra.run.smithi110.stdout:+hv01.front.sepi 67.205.162.81 3 u 40 64 377 0.064 1.009 0.684 2024-09-15T03:42:16.442 INFO:teuthology.orchestra.run.smithi110.stdout:+hv02.front.sepi 63.231.80.2 3 u 38 64 377 0.091 1.276 0.680 2024-09-15T03:42:16.442 INFO:teuthology.orchestra.run.smithi110.stdout:*hv03.front.sepi 74.6.168.72 3 u 46 64 377 0.084 1.032 0.452 2024-09-15T03:42:16.442 INFO:teuthology.orchestra.run.smithi110.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-15T03:42:16.444 INFO:teuthology.orchestra.run.smithi177.stdout: remote refid st t when poll reach delay offset jitter 2024-09-15T03:42:16.444 INFO:teuthology.orchestra.run.smithi177.stdout:============================================================================== 2024-09-15T03:42:16.444 INFO:teuthology.orchestra.run.smithi177.stdout:+hv01.front.sepi 67.205.162.81 3 u 37 64 377 0.101 -0.868 0.552 2024-09-15T03:42:16.444 INFO:teuthology.orchestra.run.smithi177.stdout:+hv02.front.sepi 63.231.80.2 3 u 36 64 377 0.087 -0.499 0.577 2024-09-15T03:42:16.444 INFO:teuthology.orchestra.run.smithi177.stdout:*hv03.front.sepi 74.6.168.72 3 u 44 64 377 0.085 -1.274 0.820 2024-09-15T03:42:16.444 INFO:teuthology.orchestra.run.smithi177.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-15T03:42:16.444 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-15T03:42:16.453 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-15T03:42:16.453 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-15T03:42:16.462 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-15T03:42:16.469 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-15T03:42:16.477 INFO:teuthology.task.internal:Duration was 839.976666 seconds 2024-09-15T03:42:16.477 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-15T03:42:16.571 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-15T03:42:16.571 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-15T03:42:16.574 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-15T03:42:16.625 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-15T03:42:16.625 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi110.front.sepia.ceph.com 2024-09-15T03:42:16.625 DEBUG:teuthology.orchestra.run.smithi110:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-15T03:42:16.639 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi177.front.sepia.ceph.com 2024-09-15T03:42:16.639 DEBUG:teuthology.orchestra.run.smithi177:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-15T03:42:16.652 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-15T03:42:16.652 DEBUG:teuthology.orchestra.run.smithi110:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-15T03:42:16.654 DEBUG:teuthology.orchestra.run.smithi177:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-15T03:42:16.689 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-15T03:42:16.690 DEBUG:teuthology.orchestra.run.smithi110:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-15T03:42:16.735 DEBUG:teuthology.orchestra.run.smithi177:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-15T03:42:16.807 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-15T03:42:16.817 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-15T03:42:16.817 DEBUG:teuthology.orchestra.run.smithi110:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-15T03:42:16.831 DEBUG:teuthology.orchestra.run.smithi177:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-15T03:42:16.846 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-15T03:42:16.856 DEBUG:teuthology.orchestra.run.smithi110:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-15T03:42:16.858 DEBUG:teuthology.orchestra.run.smithi177:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-15T03:42:16.871 INFO:teuthology.orchestra.run.smithi110.stdout:kernel.core_pattern = core 2024-09-15T03:42:16.872 INFO:teuthology.orchestra.run.smithi177.stdout:kernel.core_pattern = core 2024-09-15T03:42:16.902 DEBUG:teuthology.orchestra.run.smithi110:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-15T03:42:16.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T03:42:16.934 DEBUG:teuthology.orchestra.run.smithi177:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-15T03:42:16.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T03:42:16.940 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-15T03:42:16.951 INFO:teuthology.task.internal:Transferring archived files... 2024-09-15T03:42:16.951 DEBUG:teuthology.misc:Transferring archived files from smithi110:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905284/remote/smithi110 2024-09-15T03:42:16.952 DEBUG:teuthology.orchestra.run.smithi110:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-15T03:42:16.977 DEBUG:teuthology.misc:Transferring archived files from smithi177:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905284/remote/smithi177 2024-09-15T03:42:16.977 DEBUG:teuthology.orchestra.run.smithi177:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-15T03:42:17.006 INFO:teuthology.task.internal:Removing archive directory... 2024-09-15T03:42:17.006 DEBUG:teuthology.orchestra.run.smithi110:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-15T03:42:17.008 DEBUG:teuthology.orchestra.run.smithi177:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-15T03:42:17.014 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-15T03:42:17.024 INFO:teuthology.task.internal:Not uploading archives. 2024-09-15T03:42:17.024 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-15T03:42:17.036 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-15T03:42:17.036 DEBUG:teuthology.orchestra.run.smithi110:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-15T03:42:17.059 DEBUG:teuthology.orchestra.run.smithi177:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-15T03:42:17.064 INFO:teuthology.orchestra.run.smithi110.stdout: 924112 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 15 03:42 /home/ubuntu/cephtest 2024-09-15T03:42:17.067 INFO:teuthology.orchestra.run.smithi177.stdout: 924112 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 15 03:42 /home/ubuntu/cephtest 2024-09-15T03:42:17.069 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-15T03:42:17.078 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-15T03:42:17.100 INFO:teuthology.run:Summary data: description: rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} tasks/libcephsqlite} duration: 839.9766664505005 flavor: default owner: scheduled_skanta@teuthology success: true 2024-09-15T03:42:17.101 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-15T03:42:17.185 INFO:teuthology.run:pass